The parish of Gladsmuir derived its name from gleds or kites, birds of prey, which, in the heat of clerical debate, became disparaging symbols for Robertson himself. The Edinburgh–Berwick post road (now the A1) passes along a stony ridge running east to west across the middle of the parish. From the kirk situated by the post road, one could see down the gentle slope to the Firth of Forth on the north and down the equally gentle slope to the Tyne River valley to the south. The chief natural resource was coal, which had been mined in the western portion of the parish for centuries. Those not employed in the mines were engaged in agriculture. Near the kirk, a brickwork operated that may have brought his grandfather to Robertson's mind. There were no towns in the parish and besides Gladsmuir just a handful of villages: Longniddry, Penston, Samuelston, and Macmerry. The kirk itself dated just from 1695, though it now forms only picturesque ruins. But it was an interesting structure, featuring crow-step gables, an elegant bellcote, and two galleries inside the church. Built in 1725, the manse was the second for the parish. It was now near the church (the former manse was a half mile away in Penston) and by virtue of its recent construction was probably still in good condition, although the minister's glebe was six and one half acres of “indifferent land.” In about 1750, the minister's income was £69 (£53 in victual and £16 in cash) plus the value of the glebe and pasture, making a total of £75, modest when compared with Hugh Blair's £103 and Carlyle's £86.

The parish population was mixed. On the one hand, there were distinguished figures like William Law of Elvingston. He was a very active heritor, and Robertson often acted in concert with him. His father had been professor of moral philosophy at Edinburgh, and he himself had a distinguished career in law, serving as sheriff of Haddingtonshire for fifty years and holding a prominent place in the Scottish bar. He was also an important agricultural improver, becoming one of the first to practice harrowing wheat in the spring months.