Born in 1888, the son of an Irish Catholic father and an English Baptist mother, John Thomas Murphy was brought up with an older and a younger sister in a ‘back-to-back’ terraced house in Wincobank, a small village on the outskirts of Sheffield. The family was poor, not least because of his father's drinking bouts and habit of leaving his work as a blacksmith's striker to take an alcoholic tramping holiday for several weeks each spring. To supplement the family income, Murphy's mother took in lodgers and baked and sold bread and cakes, which, from the age of seven, young Jack was given the task of hawking from door to door before and after school. He also worked for a local farmer on a milk round each morning and evening throughout the week, which enabled him to buy clothes which the family could otherwise not afford. It seems to have been a relatively happy childhood despite the fact his various jobs and strict Methodist upbringing meant he was only able to snatch half an hour here or there to play with his friends. Sundays were a particularly busy day:

The young Jack Murphy received his elementary education at the Wincobank Council School and took his studies seriously, although the formal teaching of the ‘three Rs’ appears to have been designed above all to turn out ‘a good Christian boy who would be ready and anxious to be a good wage worker’. Like many other children of the period, he left school at the age of 13: in Murphy's case to work at the Vickers engineering factory in Brightside, Sheffield. He constantly agitated to be moved around the different workshops to get an all-round apprenticeship training, with the result that in the space of a few years he had worked on almost every machine and on all classes of work in the factory. His original ambition was to eventually get out of the plant and enter the Civil Service.