Pepperhill had two capital mansion houses, Upper Pepperhill and Lower Pepperhill. The latter, ‘a handsome sashed house’ was built by Gilbert Talbot (1672–1743), 13th Earl of Shrewsbury, who had become an English Jesuit and held the Italian bishopric of Bertha before he succeeded his relation, the 1st and last Duke of Shrewsbury and the 12th Earl of that title, in 1718. He is said to have ‘erected a brick mansion, with a court before it’ where, after a life of piety and seclusion he died. His house was used as a private school in the early years of the nineteenth century but was demolished in 1853 and two cottages erected on its site using salvaged materials.

The remaining house, which bears the Pepperhill name, stands upon a cliff edge and was a hunting lodge set in the Talbots’ Albrighton deer park that had been first enclosed in 1519. Albrighton had come to the Talbots through the marriage of John Talbot – High Sheriff of Shropshire in 1528 – with the heiress of Adam Troutebeck of Mobberley in Cheshire. Leland noted that ‘Sir John Talbot that married Troutbek’s [sic.] heir dwelleth in a goodly lodge on the high top of Albrighton Park’. Pepperhill was supposedly a house to which Mary Queen of Scots was brought by the Earl of Shrewsbury.

Although the core of the house is of early sixteenth-century date, most of Pepperhill dates largely from 1699 when it was rebuilt by William Hill, the steward to the Talbot Earls of Shrewsbury, following the collapse of three of the chimney stacks in 1698, an accident which killed three men. Hill described how in 1699 he ‘built the above said house, & came with my family to live in it at our Lady day 1700’. Pepperhill is now of red brick above a sandstone plinth and presents a front of four bays and two storeys, with a centre bay with massive projecting chimney stacks.

Pepperhill was acquired following an exchange from the Earl of Shrewsbury by the Earl of Dartmouth of nearby Patshull, across the border into Staffordshire.