Brougham Castle was founded by Robert I de Vieuxpont in the early 13th century. The site, near the confluence of the rivers, Eamont and Lowther, had been chosen by the Romans for a Roman fort called Brocavum. The earthworks of the fort can still be seen to the South of the Castle.