CAMPSEA ASHE LIES TWO MILES EAST OF WICKHAM MARKET and Ashe Common and provided the site for High House (so called, apparently, on account of its four storeys), which William Glover, a servant of Thomas Howard, Earl of Suffolk, built there in the early seventeenth century. This was on the estate that William's father, John Glover, a servant of Thomas Howard, third Duke of Norfolk, had acquired from Thomas Goodwyn and others in the reign of Elizabeth I. The Glover family did not remain long at the house as William Glover was killed in 1641 at the White Hart in Wickham Market, and in 1652 his son sold the property to John Sheppard whose family came from Mendlesham, some miles to the west of Campsea Ashe. The Sheppard family retained the estate for almost 250 years, enlarging the original brick built house and, in the early nineteenth century, building the stable block.

The Sheppards were a prominent gentry family, and various members served the office of High Sheriff of Suffolk. In addition to the Campsea Ashe estate they also owned the nearby Bawdsey estate. John George Sheppard, the last of the family to own High House, was a member of the jury at the first trial for perjury of the impostor Arthur Orton, who claimed to have succeeded to the Tichborne baronetcy. Indeed, the strain imposed on Sheppard by his attendance at the year-long trial is said to have broken his health. Following his death in 1882, leaving a widow but no children, High House and its estate were sold. The property was auctioned in London in July 1883 and, the bidding having started at £50,000, it was sold to the Hon. William Lowther of Lowther Lodge, Hyde Park, a brother of the third Earl of Lonsdale, for £105,000 (the equivalent of over £5,000,000 today).

THE HOUSE that William Lowther purchased in 1883 was materially different from that which William Glover had built in the seventeenth century. A fire in 1864 had severely damaged the original house and its later additions. John George Sheppard had employed the architect Anthony Salvin to rebuild the house after the fire.