Long the seat of the Lutwyche family, the first recorded being Roger, Lord of Lutwyche, of whom the fourth in descent was William Lutwyche, living in 1418. Throughout the sixteenth and seventeenth centuries the family distinguished themselves in the legal profession and grew rich through its practice. Their house, in the early sixteenth century, was apparently a gabled structure of coursed rubble limestone, of which two stout gable ends survive on the north-west front of the present house. In 1587, though, the house was rebuilt by Richard Lutwyche in red brick with blue-brick diaper patterning, to produce an E-plan mansion, with three gable ends spaced by low square turrets that face south-east. Lutwyche’s eldest son, Edward Lutwyche (d. 1614), was admitted to the Inner Temple in 1565 and his brother John, an attorney of Lincoln’s Inn and executor of Judge Owen of Condover’s will, was responsible for rebuilding Shipton Hall (q.v.) and the chancel of Shipton Church.

In the middle years of the eighteenth century, the house was modernised to the designs of an unknown architect, with a new pedimented, brick stable range built to the north-east. The stables face south-west and a recently discovered watercolour by Moses Griffith, of Lutwyche in 1793, reveals that the emphasis of the house itself was south-west facing at this time. Griffith shows the three sixteenth-century gables on the south-east front, but the west side is depicted with a slightly projecting centre of three bays surmounted by an open pediment on scrolls and containing a Diocletian window. A favourite of William Baker of Audlem (1705–1771) and the Hiorne brothers, David (1715–1758) and William (c. 1712–1776), this window raises the possibility of these names being associated with Lutwyche although confirmatory evidence appears to be lacking.

The work at this time, though, appears to have extended to the interiors including a new staircase hall which formed a part of a brick extension on the north-west front of the house. The staircase itself rises around three sides of a hall created in the angle between library and main hall.