It is not known where Robert and Mary Home settled after his retirement: he may have inherited Sharplaw on his father's death in 1743, but it is doubtful whether he ever lived there. However, it is thought that they were already living in York by the time of the birth of their second daughter Mary on 2 May 1748, and also when their first son William (who died in infancy) was born in 1749. By 1750 Robert was practising as a private surgeon in Hull, and it was there that their other six children were born: Hutcheson on 7 July 1750, then Robert ‘Bob’ on 17 August 1752, William on 21 November 1753, followed by twins Helen (who died in infancy) and Elizabeth on 18 September 1754 and Everard on 6 May 1756.

At some date between 1758 and 1763 the Home family moved to Scotland. There they lived with an aunt, who may have been Robert's widowed sister, Elizabeth Jacks, who lived in Jack's Land in the Canongate in Edinburgh's Royal Mile. Anne's distant cousin, David Hume, also lived in an apartment in Jack's Land from 1753 to 1762, where he wrote nearly the whole of his History of England. Anne probably went to school in Edinburgh, perhaps like Walter Scott's mother, Anne Rutherford, at Mrs Euphemia Sinclair's, whose ‘young ladies were in after life fond of reading, wrote and spelled admirably, were well acquainted with history and the belles lettres, without neglecting the more homely duties of the needle and the accompt book; and perfectly well bred in society’. Or she might have gone on ‘to be finished’ at the Hon. Mrs Oglivie's, to be taught deportment, dancing and, most importantly, music. In Edinburgh the only instruments played by women were the harpsichord, spinet and the cittern or ‘English guitar’, all harmonically self-supporting so that musical performances were individual and held in private houses. However, the concerts of the Edinburgh Musical Society were gradually infiltrated by ladies and became fashionable events, especially after the opening of the Society's new rooms at St Cecilia's Hall in 1762, built to designs sent from London by Anne's future brother-in-law, Robert Mylne.