Ludford House stands immediately to the south of Ludlow in what, until 1901, was a part of Herefordshire, and is separated from the town by the River Teme. A fifteenth-century bridge leads out of the town to the village and the road climbs immediately to the west side of the four stout projecting chimneys of Ludford House. These intimidating sentinels act as a prelude to the stone archway, with the close studded, timbered upper storey and gable end, that marks the entrance to the house courtyard. On the opposite side of the road is the former deer park whilst, embracing the house, to the north-east, stands the parish church of St Giles, which now contains the monuments to members of the Foxe and Charlton families, whose seat Ludford House once was.

In the thirteenth century, there was a leper hospital on the site of Ludford House, annexed to the Church of St Giles which, at that time, was a chapel of Bromfield Priory. This later became an almshouse for the church but at the Dissolution of the Monasteries, when Bromfield Priory’s possessions were sold off, the site passed initially to John, Earl of Warwick. In 1536 Lord Warwick sold Ludford to William Foxe (d. 1557) who, with his wife, the Ludlow heiress Jane Donne, appropriated the site for their own house and built a new almshouse to the north-east of the church, known as St Giles’ Hospital. Parts of Ludford House, namely the west range, are said to remain from the time of the Leper Hospital, although most of what one sees today dates from the Foxe family ownership and later. Ludford is, essentially, a great quadrangular building, with a courtyard at its centre which now provides the entrance to the building. The range to the south was originally stabling and the farmery, the west range – the supposed former Leper Hospital site – largely given over to the kitchen and service use, whilst the east and north ranges provided the domestic accomodation of the owner. At the inner north-east angle of the courtyard is a structure, now a gabled timber-framed range, which is thought to have been the courthouse.