Although Ludlow could not carry out his own programme, he was no longer isolated as he had been before he met the other Christian Socialists. He had friends, and his allegiance, once given, was usually lifelong. These friends kept him in the centre of that intellectual aristocracy of the rising middle class which stamped its character on the latter half of the nineteenth century. His continued association with Maurice, Hughes, Kingsley and others ensured that he was at the intersection of a series of interlocking groups, concerned with a wide range of activities, in many of which he played a distinctive role.

In spite of differences of opinion about the policy to be adopted in the Christian Socialist movement, he had remained on friendly terms with Tom Hughes. The two men, indeed, shared a house together for seven years. In 1852 Ludlow decided to leave Cadogan Place off Sloane Square and to build a house for himself and his mother in Wimbledon, then still rural and only linked by a branch railway line to London. His mother's health was being affected by London air, a factor in persuading him to leave London and to abandon Christian Socialism. Tom Hughes also wanted a country home for his growing family. The two men decided to co-operate. They managed to purchase a portion of land at Copse Hill, part of Lord Cottonham's estate, which then came into the market.