Despite appearances to the contrary, Pitt was in part a political outsider, though this was more a matter of temperament than birth. His paternal background offered an instructive example of the flexibility of the English ancien régime, its ability to absorb new wealth and rising men. Pitt's paternal background might appear undistinguished, though the Pitt clan contained solid gentry, even eminent local magnates of elite status at Strathfieldsaye in Hampshire and Encombe in Dorset. Pitt's grandfather, the masterful and determined Thomas Pitt of Boconnoc (1653–1726), known as ‘Diamond’ Pitt because of a famous diamond he brought back from India, was the son of a Reverend John Pitt, a Dorset cleric, whose brother was Mayor of Dorchester, whose uncle was a MP and prominent official under James I and whose grandfather was a wealthy official of Elizabeth I. ‘Diamond’ Pitt made a famous fortune in India, much of it gained while opposing the interests of the East India Company; this he invested in the purchase of English estates, some of which brought electoral influence. In 1691 he bought Old Sarum, the site of a depopulated medieval borough near Salisbury, from the trustees of the Earl of Salisbury and he sat in Parliament for that seat, Salisbury, or Thirsk in 1689–98, 1710–16 and from 1717 until his death in 1726.

‘Diamond’ Pitt's wealth launched his children into prominence. His eldest son Robert (c. 1680–1727) was William's father.