The Impossible Office?

Anthony Seldon

doi:10.1017/9781009429740.011

Eight under-accomplishing premierships since Thatcher stood down in 1990, and a long list of long-standing domestic problems not resolved by successive administrations, including stalled productivity in the economy, crumbling transport infrastructure and chronic housing shortages, repeated failures to address social care and a failing health service, speak of a failure of leadership at the top. Many PMs – not all– who stepped up were potentially well-equipped for the job. So what has happened? Has the job now become impossible: or is it the quality of the incumbents and their preparedness for office that have declined? This final chapter seeks to provide answers. It probes the issue of quality, and examines which prime ministers have been successful and why, assigning them to one of six grades. It examines how judgements about premierships are formed, and the role of the individuals themselves in shaping those perceptions, before concluding with five proposals to ensure that, as we move towards the mid twenty-first century, prime ministerial performance might improve.

The office of prime minister is no longer working suffciently well. Our exploration of three hundred years of prime ministers, and the experiences of the fifty-seven different incumbents, shows the deterioration, especially this century.

Eight under-accomplishing premierships on the trot since Thatcher stood down in 1990, and an unforgivable list of urgent domestic problems in 2024 not resolved by successive administrations, including stalled long-term growth and productivity in the economy and growing inequality, the absence of a coherent and sustained strategy to benefit from Brexit, crumbling infrastructure and chronic housing shortages, repeated failures to address social care and a struggling health service, speak of a breakdown of leadership at the very top. Many PMs – not all – who stepped up were potentially equipped for the job. So what has happened?

Has the job now become impossible: or is it the quality of the incumbents and their preparedness for office that is the issue? This final chapter seeks to provide answers and solutions. It probes the issue of quality, and examines which prime ministers have been successful and why, assigning them to one of six grades. It examines how judgements about premierships are formed, and the role of the individuals themselves in shaping those perceptions, before concluding with proposals to ensure that, as we move towards the mid twenty-first century, prime ministerial leadership improves.

Premierships are Not Pop Songs or Films

What was the greatest pop song of all time? The best film ever? Many of us have our own lists and passionately defend our judgements. Queen’s Bohemian Rhapsody or Taylor Swift’s Shake it Off? Citizen Kane or Parasite? Ranking the fifty-seven prime ministers from ‘best’ to ‘worst’ too has become a favourite and equally harmless (if meaningless) national preoccupation. Walpole or Pitt the Younger? Attlee or Thatcher? It’s anyone’s guess.

An end-of-century poll by BBC’s Radio 4 in 1999 of twenty prominent historians, politicians, and commentators decided that Winston Churchill had been the ‘best’ prime minister over the previous hundred years, followed by Lloyd George and Clement Attlee. A different panel of judges was used for a 2004 survey by the University of Leeds and MORI, in which 139 academics specialising in twentieth-century British history and politics were asked to rank on a scale of one to ten how ‘successful’ or ‘unsuccessful’ they thought each twentieth-century prime minister had been in office. Attlee came first on this table with a mean score of 8.34, followed by Churchill, Lloyd George, and Thatcher, with the three Liberal prime ministers achieving a mean average of 6.18, Labour prime ministers 5.81, and the twelve Conservative prime ministers last with 4.81.¹

The notion of ranking had taken hold, so much so that in the lead-up to the 2010 general election, The Times constructed a poll of all British prime ministers, with rankings given by correspondents at large, and by three specialists on the newspaper’s in-house team, Ben Macintyre, Matthew Parris, and Peter Riddell. On this wider survey, Winston Churchill still came top, with Lloyd George second and Gladstone third, Pitt the Younger fourth, Thatcher fifth, and Peel sixth. John Major was one of the biggest risers, from close to the bottom of the twentieth-century tables to almost middle, twenty-eighth out of fifty-two, showing how perceptions of prime ministers can vary once they leave office.² A 2016 University of Leeds survey of eighty-two academics, focusing just on post-1945, saw the usual suspects in the top three berths, but another ‘improved’ prime minister, Macmillan, climbing to the fourth slot.³ Finally, Iain Dale, who edited a collected volume on prime ministers in late 2020, compiled a table with five of his authors of all prime ministers since 1721, the top six being Churchill, Gladstone, Pitt the Younger, Thatcher, Attlee, and Lloyd George with, at the other end, Compton, Devonshire, Canning, and Goderich, nudging Eden off his bottom slot in the earlier tables.⁴ After Truss’s mayfly premiership in 2022, Dale’s selectors may now view the bottom five differently.

Peter Hennessy, to whom this book is dedicated, and doyen commentator on the office of PM, moves us away from La La Land (a state of mind, not the film) and grounds his ranking in actual descriptors of what the prime ministers have done. He developed what he called a ‘crude taxonomy’ to rank post-1945 prime ministers.⁵ Attlee and Thatcher were the two to reach his ‘very top flight’, as the two ‘weathermakers’ of their time, and suggested Thatcher was the greater because she forged a new consensus, unlike Attlee who built on an already dominant postwar consensus forged in World War Two. Blair and Heath ranked in his second tier of ‘nation- or system-shifters’ for constitutional reforms and EU accession respectively, failing to make the top flight. Major was, unfairly in hindsight, forced to share the bottom of the pile with Eden.

Rankings of US Presidents, described by a passionate advocate of their value, Robert W. Merry, as ‘a substantial body of thought on presidential performance … the closest we can come to history’s judgement’, has perhaps more value. Presidents are in office for fixed terms, fewer have served (45 to date) and they are more likely to have completed a full term (only three served less than 2 1/2 years) than prime ministers. Their records in office are more widely known. But the exercise is still flawed.⁶

Such lists are entertaining, but they tell us little. Do not expect the pictures of the prime ministers that hang on the staircase in Number 10 to be reordered according to the latest ranking any time soon. So why cannot we order prime ministers with anything like the same precision we rank the ‘greatest’ films, songs, or novels of all time? We can see the films, listen to the songs, and read the books today in the present moment, and can compare them, taking account of the different periods in which they were created. But we cannot go back and re-experience the premierships, any more than we can compare great stage actors like Sarah Bernhardt or Henry Irving with today’s greatest like Maggie Smith or Ian McKellen, because their performances cannot be recreated. It is not even possible to rank sports people reliably over time, because even where, as with cricket, there is some comparable data, we cannot say W. G. Grace or Donald Bradman was ‘better’ than Viv Richards, Joe Root or Ben Stokes, because they were playing at such different times, where the frequency of matches, the equipment, and training were so different – akin to comparing Jonny Bairstow’s explosive play to fellow Yorkshire batter Geoffrey Boycott’s inert stoicism.

Comparative judgements on prime ministers are weakened further by our inadequate knowledge of all but the most recent incumbents, plus a sprinkling of others, Gladstone, Lloyd George, Churchill, and Attlee. Even historians who know a great deal about say the eighteenth century often lack the knowledge to make meaningful comparison with more recent prime ministers. Personal opinions are bound to weigh too, as is perhaps revealed in the Leeds and Times polls.⁷ But lists are rendered insubstantial most of all by the absence of agreed criteria on what constitutes ‘success’ for a prime minister. Many people know what a great novel or painting looks like, but a great premiership? How can one rank those whose challenges differed over time? Besides, prime ministers do not begin on a level playing field. Much should be expected from those to whom much is given. Blair came to power in May 1997 with a mouth-watering majority, a Labour Party and trade union movement united behind him, with a strong economy, and on a tide of popular and intellectual support. How can we rank him against his predecessor John Major, who came to office when the party had already been in power for over eleven years, was tired and deeply divided over Europe, with an economy in trouble, and a small majority steadily being chiselled away in by-elections? A much better comparison for Major is the performance in office of Brown or Sunak.

In place of crude rankings, we offer six designations which describe what prime ministers actually did and achieved within their historical contexts. Not every PM had the opportunity to shine; some who had the opportunity did so only dimly; others triumphed in the most unpropitious of circumstances. Within these broad categories, we cannot meaningfully say any one prime minister was ‘better’ than any other. Assessing them in this way has another advantage: it points the way to understanding how the office can be improved in the future. Britain needs more high-performing prime ministers, and fewer losers: the country deserves better.

Tier I: Agenda Changers.

Top-tier prime ministers, as we saw in Chapters 3 and 4, rose to the historic challenges of their period in power, won notable general elections, changed the course of the country, and with it, the way the job of prime minister operated. They either raised the standing of the country internationally, or bolstered the Union, key tasks for any prime minister, or both. More than merely tribal leaders, they had a sense they were leading the whole nation, and their influence was felt for many years after them, with successor prime ministers operating like them or, more often, deliberately choosing to be unlike them, but none escaping their shadows. They are: Robert Walpole, William Pitt the Younger, Robert Peel, Viscount Palmerston, William Gladstone, David Lloyd George, Winston Churchill, Clement Attlee, and Margaret Thatcher.

A similar list of nine top-tier US presidents might include George Washington, Thomas Jefferson, Abraham Lincoln, Woodrow Wilson, F. D. Roosevelt, Harry Truman, John F. Kennedy, L. B. Johnson, and Ronald Reagan. As in the United Kingdom, there hasn’t been one such figure for over thirty years. All these US presidents were weather-makers who changed the direction of their country positively.

The nine top British premierships provide the answer to one of the questions we posed at the outset of the book: how has the office survived over 300 years? What these top-tier prime ministers did was to reinvent the office for each successive new age. They renewed it. After the seventeenth century, there was no foreign invasion, nor revolution in mainland Britain, nor civil war sweeping the PM aside. The monarchy equally, scarred by recent history, had by the beginning of the eighteenth century ceded power peacefully, if unevenly, to the PM. These prime ministers thus made the political weather and altered the framework of politics – shifting the so-called ‘Overton window’, i.e. the notion of what had hitherto been considered politically possible. Walpole is in the top tier because he defined the job by his impressive range of activity in it, and his longevity in office (twenty-one years, the record), and embedded the idea of the prime minister as a part of the constitutional structure. For all that, the nascent office could easily have disappeared over the following twelve premierships and forty years. It needed a brilliant operator, a prolonged bout in the office, and a meaty task to do, if the office was to survive into the next century. Up steps the inordinately young Pitt the Younger, a genius financially, administratively, and diplomatically. Pitt still largely inhabited the claustrophobic Westminster world of Walpole’s politics. It was not a given that the office would survive the emergence of political parties. Peel, the first Tier-1 prime minister after the 1832 Reform Act to head a recognisably modern political party, a major addition to the tasks of the job, steered the office through the transition. Further, by splitting the prime minister off from the Chancellorship and the Treasury, he sculpted the shape of the modern office.

The nineteenth century saw Britain’s emergence as the dominant global power. In earlier periods of ascendancy abroad, it had been the monarch, Edward III, Henry V, or Elizabeth I, who had personified the country and its foreign policy. Palmerston, immersed in foreign policy for longer than any incoming prime minister in history, managed to embed the office at the head of the country’s global ambitions while managing to be the most popular and known prime minister in history so far that century. As Palmerston extended Britain’s reach across the world, Gladstone extended the reach of the prime minister across the domestic life of the nation, modernising the apparatus of central government in doing so. The prime minister was now the undisputed head of their party across the country, offering a programme for government to be decided by the electorate at a general election, charged with enacting it if successful.

The prime minister now had the ambition, but not the apparatus within Downing Street to service that ambition. Gladstone and his immediate successors had to get by with puny staff in No. 10.

Lloyd George created the modern prime minister‘s office when he set up the cabinet secretariat in December 1916 and expanded the size of No. 10 to manage the volume of work during and after the war. Not since Peel had dispensed with the Treasury did a prime minister have so much firepower at their personal disposal. Lloyd George was a giant who dominated British politics and policy for nearly twenty years, oversaw the division of Ireland, and presided over the reshaping of Europe, and the world, at the Treaty of Versailles.

Winston Churchill was the supreme war leader of Britain at its greatest peril in the 300 years. Britain’s standing was incomparably higher in 1945 than it had been in 1940. He returned to No. 10 in 1951 after winning the general election, and crafted an underestimated ‘Indian summer’ administration until 1955. He was the most unusual of the nine on our list, in that he changed the perception of the possibilities of the office, but didn’t change it physically. Labour’s first prime minister to qualify, Clement Attlee, was one of the more technically skilful incumbents in history, presiding over the most policy-heavy government in history and transitioning the country to a national economy. Thatcher equally was remarkably adept at the job, driving domestic and foreign policy with a minuscule team at No. 10, exploding the myth that prime ministerial failure can be blamed on No. 10 being too small.

Tier II: Major Contributors.

This next category sees ten prime ministers who had a decisive influence on the country, but were often sui generis, without the long-lasting mark on policy or the office of those in the top tier. And they may have been successful winning general elections, but they failed to rise to the historic challenges they faced. They are: William Pitt the Elder/Chatham, celebrated for war leadership and becoming the first popular politician in the country, but who didn’t blossom in the top job; Lord Liverpool, who brought vast experience and stability to government, oversaw the conclusion of the Napoleonic Wars and the grave unrest after it, and was one of the first to see the job as coordinating other government departments, but who didn’t modernise the constitution; Earl Grey, who presided over the passing of the Great Reform Act in 1832 and the Slavery Abolition Act in 1833, which went far further than the abolition of slavery itself in 1807 to root it out, but was too briefly in office; Benjamin Disraeli, who pushed through the 1867 Reform Act as Chancellor, and when prime minister passed social welfare legislation, was the first PM to attend a conference abroad, and had an enduring impact on the Conservative Party, but who, like Chatham, did his best work and was at his best before becoming PM. In the twentieth century: H. H. Asquith advanced the social policy agenda above all with the 1911 National Insurance Act, and drove through the highly significant 1911 Parliament Act, but failed to rise to the challenge of the First World War; Stanley Baldwin, who provided stability for the country in the volatile interwar years, helped induct Labour into parliamentary democracy, marginalised extremism on left and right, and saw the country through the abdication crisis of 1936, but was not a commanding leader either at home or abroad; Harold Macmillan, the first television prime minister, who promoted decolonisation and the first attempt to join the European Community; Harold Wilson, who oversaw liberalising policies under Home Secretary Roy Jenkins, was the first prime minister since Lloyd George significantly to expand the size and reach of Number 10, and won four elections (the most ever for a Labour leader) but who, like Macmillan, did not successfully tackle Britain’s chronic economic problems, with Wilson’s key initiatives ‘In Place of Strife’ and the Department of Economic Affairs both falling in 1969; Edward Heath, for taking the country into the European Economic Community; and Tony Blair, for constitutional reforms, including regional devolution, an elected Mayor of London, and the Supreme Court, social and economic reforms, including the minimum wage and the invention of tax credits, as well as being the first Labour prime minister to win three successive election victories, but who did not find an enduring solution to Britain’s chronically troubled relationship with the EU, did not reshape public services, and mired Britain in the errors of the 2003 Iraq invasion.

Tier III: Positive Stabilisers.

This third class of prime minister provided competent or better leadership, but without the historic acts of the ‘major contributor’ group, or introducers of important changes like the second. The opportunity for enduring change was often absent for these PMs, underlining the importance of timing: it can be difficult to make a big impact if the conditions are adverse or if a PM succeeds a giant. Sometimes, the prime ministers in this category were better leaders than those in the tier above: but the conditions were adverse. Henry Pelham was an accomplished manager of the House of Commons and of the national finances, and helped bring stability after the Jacobite uprising of 1745–6; the Duke of Newcastle was an astute financer of war; Spencer Perceval, until cut short by his assassination, governed well in the face of economic depression and Luddite unrest, and successfully oversaw the conduct of the Peninsula War; the Duke of Wellington piloted through Catholic emancipation; Lord Salisbury provided stability at the end of the nineteenth century, introduced incremental reform including county councils, steered the country peacefully in Europe, and, more controversially, expanded the Empire. Whilst undoubtedly a success on his own terms, he was responsible for few fresh initiatives over his fourteen years, at a time of rapid change nationally and internationally. In the twentieth century Arthur Balfour oversaw social reform, strengthened British defence, and negotiated the Entente Cordiale with France in 1904 before fizzling out; Henry Campbell-Bannerman won a landslide in 1906 and introduced state pensions and free school meals, paving the way for later social reforms, but was clearly overshadowed by Asquith and Lloyd George; Ramsay MacDonald brought Labour to power in its first two governments, showing the way for organised working-class and Labour support to play its role in Westminster politics, and oversaw steady reforms in the first four years of the National government after 1931, but left no great policy mark; James Callaghan maintained stability for three turbulent years without a majority, and negotiated the IMF loan in 1976; John Major, at the top of this list, provided steady leadership after the pyrotechnics of the Thatcher decade, allowed her free market reforms to embed whilst taking them forward in his own style, strengthened the economy, and found a temporary way for Britain to accommodate itself to the EU. In the twenty-first century, Gordon Brown piloted Britain strongly through the Global Financial Crisis; David Cameron steadied the economy for six years during Britain’s first peacetime coalition government since the 1930s, innovated with the National Security Council, while introducing some liberal reforms, notably same-sex marriage; and Sunak provided stability after Johnson and Truss.⁸

Tier IV: Noble Failures.

The next class is ‘noble failures’, prime ministers who tried to do the right thing, were principled and dedicated, but became overwhelmed by the events they faced at the top. These five prime ministers come from each of the four centuries, with two in the nineteenth. Lord North provided strong financial leadership for his first five years in office, was a good House of Commons and elections manager, excelled at overseeing the finances to pay for the American war, and helped maintain political support in its early years till overwhelmed by its reversals; Lord John Russell was a passionate and effective reformer earlier in his career, and helped found the modern Liberal Party, but he failed as prime minister to deal with the Irish Famine, and mishandled the politics of reform, leaving the Conservatives to pass the 1867 Reform Act; Lord Derby, another figure who promised much and helped found the modern Conservative Party, was unable to achieve what he wanted in his brief periods as prime minister except the franchise reform not achieved by Russell; Neville Chamberlain, who had been such a vital reformer and successful administrator before he reached Number 10, was so desperate to avoid a war against strong adversaries in Germany, Japan, and Italy, that he placed too much faith in his ability to secure an agreement; and Theresa May, who fought with almost superhuman energy and tenacity to achieve a Brexit on which her party could agree, but lacked the strategic clarity or interpersonal skills that prime ministers need if they are to succeed, and left with few of her ambitions for tackling burning injustices realised.

Tier V: Ignoble Failures.

Nine prime ministers fall into this class, all of whom lacked a basic moral seriousness, or leadership ability, or both. George Grenville had little to show for his two years beyond extending the Stamp Act to the American Colonies, which inflamed them, and prosecuting radical protester John Wilkes; The Duke of Portland saw the biggest gap between periods in office for a prime minister (twenty-four years), but lacked the qualities to be a leader as either a young or an older one, while providing competent cover for more dominant figures to do so; Henry Addington negotiated the unsuccessful Treaty of Amiens in 1802, but failed to command authority in either House of Parliament, leading to his fall; Lord Melbourne had little to show for his six years in power beyond inducting Victoria, while his louche style and involvement in scandals did nothing to add to the office; Lord Aberdeen; who was unable to provide effective leadership to the pungent politicians in his mixed ministry, failed to keep Britain out of the Crimean War, and to lead it successfully; Lord Rosebery lacked gravitas, failed to build on Gladstone’s legacy, to give a clear direction, and led the Liberals into a defeat; Anthony Eden, a truly tragic case, principled and proud, but stubborn and naive, and who led the country into the disastrous Suez campaign. Few prime ministers have fallen so low from such a height. The final two, though, plumb new depths in the prime ministerial pantheon. Johnson will be remembered for achieving Brexit if failing to produce a plan to exploit it, winning and squandering a landslide majority in 2019, shakily overseeing Britain’s response to the COVID-19 pandemic shakily, and speaking out in the defence of Ukraine. Why is he in this in category? Because he was totally lacking in the skills and the basic moral qualities the prime minister needs. After winning his majority, his premiership descended into a court politics of infighting, scandals, and inertia. Johnson burnt everyone the same: whether his closest allies, his deepest enemies or, most importantly, the electorate’s trust. He left no enduring legacy on domestic policy, especially when considering the size of his majority and the speed at which he fell – announcing his resignation just two years and 207 days after his landslide victory.

Truss was the most experienced PM for 30 years, she had a clear plan and a high intellect. Yet her forty-nine day premiership inflicted extensive damage to the economy and to her party. Her sins were lack of judgement, of unwillingness to listen, and zero understanding of history.

After her chaotic resignation, billons were permanently missing from the economy while the office of prime minister was diminished, her party divided and damaged and the British state left shocked.

Tier VI: Left on the Starting Line.

A final category are those whose premierships were too short to judge their performance, which includes some who had much promise and earlier achievement to their name. They are: George Canning (120 days), Lord Goderich (145 days), Andrew Bonar Law (212 days), the Duke of Devonshire (226 days), Lord Shelburne (267 days), Lord Bute (318 days), Alec Douglas-Home (364 days), Lord Grenville (1 year, 43 days), the Duke of Grafton (1 year, 107 days), Lord Rockingham (1 year, 114 days), and Spencer Compton (1 year, 120 days).

The Overall List

	18th Century	19th Century	20th Century	21st Century
I. Agenda Changers	Walpole, Pitt the Younger	Peel, Palmerston, Gladstone	Lloyd-George, Thatcher, Attlee, Churchill
II. Major Contributors	Pitt the Elder	Liverpool, Disraeli, Grey	Asquith, Baldwin, Macmillan, Wilson, Heath	Blair
III. Positive Stablilisers	Pelham, Newcastle, Perceval,	Wellington, Salisbury	Balfour, Campbell-Bann erman, MacDonald, Callaghan, Major	Brown, Cameron, Sunak
IV. Noble Failures	North	Russell, Derby	Chamberlain	May
V. Ignoble Failures	Grenville, Portland	Addington, Melbourne, Aberdeen, Rosebery	Eden	Johnson, Truss
VI. Left on the starting line	Devonshire, Shelburne, Compton, Bute, Grafton, Rockingham	Grenville, Canning, Goderich	Bonar Law, Douglas-Home

So What Makes for a Successful Premiership?

No magic formula exists for a successful premiership, any more than for a high-achieving sports team, company, or work of art. But a four-point approach developed over my writing on PMs moves us perhaps closer to understanding some common factors that make success more likely, if not guaranteed: they are individuals, ideas, interests, and circumstances.⁹

‘Individuals’ starts with the qualities of the prime ministers themselves. Our nine ‘top-tier’ prime ministers shared some common attributes. They all (bar Pitt the Younger) had long apprenticeships in which they learnt about governing, made painful mistakes, and arrived at the top with a maturity and a wisdom, even when blended with impetuosity, which strengthened that vital PM quality – judgement. They had a clear and achievable agenda for what they wanted to achieve in office which drove them forward, even if some of those ideas evolved or crystallised only after they were in power, or were thrust upon them by events, and which gave their premierships a coherence and purpose. They had a moral seriousness about their work, despite some, like Walpole, Palmerston, and Lloyd George, being far from moral in their personal conduct. Finally, they all possessed an iron will, bolstered by an intense work ethic and drive to get the job done. To these may be added possession of many of a range of skills discussed in Chapter 5, namely the ability to persuade colleagues in Cabinet and Parliament to get behind them, to communicate effectively near and far, abnormally high energy levels, robust physical and mental health, genuine intellectual depth and agility, an equable temperament, and not least the ability to be coldly ruthless when required, sacking ministers, dumping policies, and changing direction.

Premierships are not solo acts: the prime minister is captain of the team, and to be successful, they need senior ministers who are experienced or willing to learn, and who are skilful, determined, driven, loyal, and prepared to do the work for them. Few Cabinets from 1721 were more talented than Asquith’s peacetime government of 1908–14, featuring Lloyd George, Haldane, Grey, McKenna, and Churchill. Clement Attlee’s from 1945–51 comes close, with the wartime-blooded Bevin, Morrison, Dalton, and Cripps joined by Bevan and Gaitskell. Even with a Cabinet overflowing with capable ministers, famously the Ministry of All the Talents (1806–7), there is no guarantee of success. Gordon Brown tried to repeat the formula with his ‘Government of All the Talents’ (GOAT) by bringing five non-Labour experts into the government, albeit at a junior level, and not conspicuously successfully. None of the top nine lacked top Cabinet talent, and they sometimes faltered when that talent began to drop away, for example Gladstone and Attlee. Inside Number 10, the PMs need just three or four outstandingly able operators – the Principal Private Secretary, Chief of Staff/Permanent Secretary, Director of Communications, and Political Secretary – to lead the pack below them.

‘Ideas’ are equally essential to successful premierships. If a prime minister is fortunate to come to power on the crest of an intellectual wave, as Grey, Gladstone, Asquith/Lloyd George, Attlee, and Thatcher did, they have a head start. Lacklustre premiers scramble for ideas and go in for periodic ‘re-launches’ which are barely ever successful. Ideas mobilise, they enthuse, they bring divergent people together in common mission. The abolition of slavery was an idea, as was solving the Irish Question, imperial preference, the mixed economy, decolonisation, devolution, and privatisation.

Euroscepticism and Brexit were ideas, which proved too anarchic for May to channel, but which helped Johnson win the general election in 2019. Johnson’s ‘levelling up’ idea to tackle regional inequality gained more popular support than Major’s ‘citizen’s charter’, Cameron’s ‘big society’, or May’s ‘burning injustices’ but ultimately fell well short of lasting progress – strangled by insufficient political capital, project clarity, and money. Blair’s premiership suffered for want of an overarching idea – the ‘third way’ proved ultimately to be just a slogan.

‘Interests’ need to run with a premiership rather than against it. Pitt the Younger was able to channel the financial interests and City behind the war, while Peel ran up against powerful landed interests that resisted his attempts to repeal the Corn Laws. Gladstone’s premiership was boosted by his support from the press, grateful to him for his removal of paper duties. Asquith saw off the Lords, but was assaulted by multiple challenges from the suffragettes, trade unions, and Irish nationalists, and Baldwin’s premiership by hostility from the press and unions, while Churchill was able, in masterly fashion, to align all the powerful interests in the nation behind the war effort from 1940–5. Trade unions helped bring about the ends of the premierships of Wilson in 1970, Heath in 1974, and Callaghan in 1979, while Thatcher was able to outflank them, helping bring powerful business, financial, and media interests behind her. Declaring outright war on the Civil Service or ‘blob’ as Johnson, Cummings and Truss found was a dead end: they weakened and antagonised the very institution that they needed to deliver for them. Truss angered and upset many different interests but none as powerful as the markets, which after being damaged by the repercussions of her mini budget swiftly moved to end her premiership. Even the most powerful of prime ministers like Thatcher and Blair came across resistance from status quo elites, including judges and the professions. Interests are a fact of life in a pluralist democracy and need to be negotiated around. The art of the prime minister, as any leader, is the art of agility.

‘Circumstances’ or ‘events’ finally help explain why some premierships succeed while others fail. All the top-tier prime ministers were in office at the time of a major event – a war, an economic crisis, or an epidemic – which begs the question: do great leaders make history, or are they made, in part at least, by history? Premierships can often be defined by their success at dealing with one ‘big’ event, if they have one. Wars are the most dramatic. Walpole’s failures during the War of Jenkins’ Ear helped bring about his demise, while Pitt the Elder’s leadership in the Seven Years’ War made his name. The American War of Independence squashed North, while the French Revolutionary and Napoleonic Wars established Pitt the Younger’s and Liverpool’s reputations. The Crimean War finished off Aberdeen but elevated Palmerston. The First World War fatally damaged Asquith, as the Second World War did Chamberlain, while making Lloyd George and Churchill respectively. Attlee’s final demise in 1951 was hastened by paying for the Korean War, while Suez did for Eden, the Falklands boosted Thatcher, while Blair never recovered from Iraq. Economic downturns, as we have also seen, can impale prime ministers, as with MacDonald, Baldwin, Callaghan and Sunak. Finally, existential crises can break premierships, from the Irish Famine in the 1840s through to COVID-19 in the 2020s.

The length a prime minister serves is a critical circumstance: only twenty-five have been at the top longer than five years, and of the thirty-two who served less, it’s hard to find many of consequence; Grey is the principal exception, the twenty-eighth longest-serving incumbent, who achieved much in his three years and nine months. Truss, comparably, proves the rule. All our ‘top-tier’ prime ministers served for longer than five years: Peel being the shortest at just over that length. Longevity though is no guarantee of top-tier success, the third and fourth longest-serving incumbents, Liverpool with fourteen years and ten months, and Salisbury on thirteen years and eight months, having insufficient to show for their length of time in power. Both resisted reform.

Activism is not of course anecessary requirement for a successful premiership: Heath conspicuously did too much, with many of his policies not lasting. Johnson, similarly, was bold in rhetoric and attempted several reforms but didn’t have the temperament or time to achieve lasting success. Sometimes, the nation needs a quiet premiership.

If a sweet-spot exists of perhaps five to eight years in office, sufficient time for the prime minister to make their mark, but not enough for them to grow stale or tired, the same notion of a golden mean applies to their age. Many were too old or ill when they became prime minster, with Canning, Campbell-Bannerman, Bonar Law, and Chamberlain dying in or soon after leaving office. Many prime ministers, almost certainly a majority, were insufficiently well in office to perform at their best. They equally can be too young, including Devonshire (aged thirty-six on coming to office), Grafton (thirty-three), Rockingham (thirty-five), Rosebery (forty-six), Blair and Cameron (forty-three) and Sunak (forty-two), arguably lacking in experience for the highest office. Churchill would have been a disastrous prime minister earlier in his career: it is not impossible he might’ve succeeded Lloyd George had he succumbed to Spanish flu in the autumn of 1918. But in May 1940, he was ready, as he himself recognised. Many of the more successful were aged between fifty and sixty-five, sufficiently experienced and blooded, but still with the energy and health to make the most of the opportunity. Pitt the Younger and Palmerston are the exceptions to this rule. Pitt (twenty-four) had wisdom and political skill beyond his years, while Palmerston had remarkable health and vitality for a seventy-year-old (though would likely have been a better PM still if younger).

Winning a parliamentary majority makes all the difference to whether a premiership is successful. So it’s not surprising that all our nine ‘agenda changer’ prime ministers, and many in the second tier as well, including Grey, Baldwin, Macmillan, and Blair, were able to benefit from significant majorities. With them, the prime minister can force through controversial policies; without them, they can spend their premierships fire-fighting and scrambling for votes amongst disputatious backbenchers. The strength of the economy is another circumstance that powerfully affects success. If robust, it provides the revenue for the prime minister to do what they want to do; on the flip, high unemployment and low tax yields, even if not the incumbent’s fault, constrain them and reflect badly on them. The length of time a party has been in power can equally make or break a premiership: it is far harder to make a mark coming to office at the end of a long period of ascendancy or dominance, as Douglas-Home, Major, Brown and Sunak all found. Johnson failed to retain the confidence of his colleagues – not aided by a decade of government which had triggered division and exhaustion in the party.

Prime ministers regularly sail into Downing Street thinking that they can buck these four trends. Ignorance is bliss, and ubiquitous. The history of the last three hundred years repeatedly shows that they do so only very rarely.

Can the Verdict on a Premiership Be Improved?

If a premiership ends in acrimony, as did Peel’s in 1846, Balfour’s in 1905, May’s in 2019, and Johnson’s and Truss’s in 2022, with the party split, the initial verdict may be more negative, as it can be when prime ministers suffer heavy electoral defeats, as Baldwin’s in 1929 or Major’s in 1997. Initial verdicts on premierships though do not change much over time. Indeed, there can come a point mid-premiership, as Balfour found when Joseph Chamberlain resigned in 1903, or Major with ‘Black Wednesday’ in 1992, or Blair after Iraq in 2003, when the premiership effectively begins to unravel. The official documents in the National Archives at Kew might not be available for twenty or more years after a premiership ends, but it is already evident on the final day what it has achieved, or not, and the degree that the prime minister inspired confidence in colleagues and in the country at large.

A series of books I edited, beginning with The Thatcher Effect and concluding with The Conservative Effect: 2010–24, asked three main questions of each premiership:¹⁰ what was the state of each of the major policy areas and institutions when the prime minister came to power and what had changed at the time of their fall? To what extent was the prime minister themself responsible for initiating, driving, or supporting that change? Were the changes necessary, and successful? The last is the area that most needs the benefit of perspective, but in almost all cases, successful premierships were seen by contemporaries to be so. In the twentieth century, former prime ministers started to write their memoirs, hoping that by doing so, they would be able to ‘set the record straight’, as well as, to varying extents, settle old scores and make some money. But no prime ministerial memoir has ever significantly altered the perception of a premiership (‘memoirs’ technically are histories of a period, an autobiography just of the life, though the distinction has not always held). Asquith was the first prime minister to publish his memoirs last century, in two volumes in 1928, Memories and Reflections, 1852–1927,¹¹ followed by Balfour’s Retrospect: An Unfinished Autobiography in 1930.¹² They set the pace at the very moment, ten years after the Great War, when the nation was ready to look back. Lloyd George, not to be outdone, followed up with his six volumes of war memoirs published between 1933 and 1936.¹³ No interwar prime ministers succeeding him though published theirs – they were too ill or exhausted on quitting. Churchill wrote his subjective account of the First World War in six volumes, The World Crisis (1923–31), then wrote his six-volume The Second World War (1948–53).¹⁴ He confirmed the tradition that prime ministers could claim privileged access to consult documents from their own period in government. Anthony Eden published three volumes, relying heavily on researchers, beginning with his self-justificatory Full Circle in 1960 on the years 1951–7 (in which he excluded mention of collusion with France and Israel over Suez), followed by two volumes in 1962 and 1965 on the prewar and war periods.¹⁵

No peacetime prime minister has written at greater length than Harold Macmillan, six volumes between 1966 and 1973, although only the last three on his premiership.¹⁶ Douglas-Home wrote a congenial and light autobiography, The Way the Wind Blows, in 1976, a genre others chose not to emulate, preferring doorstops.¹⁷ Heath waited twenty-four years after leaving Number 10 before publishing The Course of My Life in 1998, which was less rancorous and more measured than had been anticipated.¹⁸ No prime minister has been quicker off the mark than his nemesis Harold Wilson, who published The Labour Government 1964–1970: A Personal Record, while he was still Leader of the Opposition in 1971, and then Final Term: The Labour Government 1974–1976 in 1979 on his last two years.¹⁹ Since him, the convention has been firmly established: detailed volumes, drawing on the prime minister’s and Cabinet official papers, and serialised in the newspapers, trumpeting ‘revelations’ (there seldom were). Six- or even seven-figure deals for the book and television rights have become almost a golden handshake entitlement of departed prime ministers, alongside lucrative speaking tours and consultancies – to compensate them for the ‘miserly’ prime ministerial salary of £165,000 (in 2022).

Thatcher produced her memoirs in 1993, Major in 1999, Blair in 2010, Brown in 2018, and Cameron in 2019, all enlightening in their different ways. None has published diaries, and none since Macmillan seems to have kept one, but their aides and ministers, with more time on their hands, have done so. Cameron, perhaps a harbinger, was regularly interviewed in quasi diary form during his premiership by journalist Daniel Finkelstein, with the transcripts later informing Cameron’s memoir For the Record. By the time a former prime minister has paid the team of researchers and drafters who help write the book, they often find there is less money left than they expected. Their memoirs are their last will and testament on the political scene. Interest in them dwindles rapidly thereafter. In 2023, Theresa May sought to buck the trend, choosing to write a more polemical analysis on leadership rather than an overt memoir titled The Abuse of Power. The work drew heavily on insights from her time as prime minister as on the Russian Salisbury poisonings and the Grenfell Tower fire, peppered with adverse asides on her successor’s conduct in office. Both Johnson and Truss similarly hope to regain lost ground with their memoirs.

Prime ministers entertain high hopes of their biographers, much as writers and artists do of theirs, and the selection of author, the content, and the degree of freedom that they enjoy, can be as contentious. ‘Official histories’, using government archives, originated with the Boer War, with the next series, on the First World War, extending to 109 volumes and concluded only in 1949.²⁰ Prime ministers, or their literary executors if they died early, emulated this tradition in appointing ‘official’ biographers, who have not usually had access to government papers, but are free to range over the prime minister’s ‘private’ or personal papers. Their prime ministerial subject is dead by the time the book is published, but their families, friends, and executors can breathe all the more intensely down their necks as they write.

Official biographies had an unhappy early experience when Oxford historian G. M. Young was asked by Baldwin himself to write his biography. When published in 1952, after Baldwin’s death in 1947, several insiders threatened to sue unless passages were removed, while historians and commentators considered the book too lightweight and insufficient to rehabilitate Baldwin after attacks, not least in the anonymous book Guilty Men (1940), which Michael Foot secretly co-authored, on his role in appeasement.²¹ In Martin Gilbert, Churchill’s family found an assiduous historian, who took over the magnum opus from Churchill’s son Randolph on his death in 1968, writing from volume three to volume eight.²² None since has written at greater length, but few have written as elegantly as Charles Moore’s three-volume official biography on Thatcher, published between 2013 and 2019. Alistair Horne wrote the two-volume official biography of Harold Macmillan²³ and Philip Ziegler single volumes on Wilson²⁴ and Heath,²⁵ but for all their undeniable merit, it’s hard to see they have altered the perception of their subject any more than (despite the hopes of Eden’s widow Clarissa) D. R. Thorpe would achieve in his biography, published in 2003. It did not rehabilitate Eden’s reputation as the figure who bestrode Britain’s mid-century foreign policy.²⁶ Ben Pimlott’s 1992 biography of Harold Wilson came as close to enhancing his subject’s stature as any before.

The verdict on a premiership is thus created, not after it is over by memoirs, biographies, and academic tomes, but by the actions of the incumbent when in office, albeit with scope for minor re-evaluation at the margins. The premierships of Baldwin, Attlee, and Major have come over better in history than they did to contemporaries at the time: while the premierships of Disraeli, Macmillan, and Blair, a little less impressive. Contemporaries tend to overestimate electoral successes, and undervalue governmental and administrative successes. However much former prime ministers might want to boost the perception of what they did, and however passionately they might believe that they were treated unfairly – by opponents, the media, colleagues – the truth is that they had their chance. If they had spent more time standing back and reflecting on their task while they were in office, read more history, took more time for reflection and honest evaluation of their work and likely impact, they might have achieved more, rather than frittering their time away, as too many have, on relaunches, reacting to news, petty squabbles, personal animosities, and vanity projects.

Has the Job Become Impossible?

The prime minister operates under heavy constraints, which have increased in the last 50 years, and still more since 2000. Three particular developments have impacted negatively on the ability of the prime minister to do their job well.

We have seen how the PM came to outshine the Foreign Secretary and monarch. But the PM has only limited control over finance and hence over the government’s agenda. The Chancellor, for long the PM’s greatest ally, has now become their biggest potential enemy. Gordon Brown significantly restricted Tony Blair’s achievements but had his own comeuppance with his Chancellor, Alistair Darling. Chancellors Philip Hammond and Rishi Sunak equally frustrated Theresa May and Boris Johnson. The prime minister is the First Lord of the Treasury, yet too often it has been the Chancellor who has acted as if they were.

Second, prime ministers have repeatedly sacrificed long-term governmental strategy for short-term political and electoral gain. PMs, from the very first, Robert Walpole, have been preoccupied with their image in the media. But the 24-hour news cycle, polling and focus groups have taken this to new levels, and the PM has too often found it difficult not to react, and obsess over headlines, the ephemeral and the short term.

Third, habits of disobedience by MPs, growing since the 1970s, have reached almost unsustainable levels. David Cameron said that social media chat among MPs made his life almost impossible as PM. WhatsApp groups, which have mushroomed since he stepped down, have made it even worse. Disloyalty by MPs has been exacerbated by No. 10’s chronic weakness at managing its MPs.

But none of these, or indeed the many other constraints, means that the job has become impossible. The blame lies principally in the way that the prime ministers conduct themselves in office. Why? The expectations for what they can achieve in office, encouraged not least by themselves, have placed almost impossible beliefs among the public of what they can accomplish. Their initial words on the doorstep of Downing Street, as we saw in Chapter 5, can reveal more about their innocent hopes and naivety about the job than the practicalities of what lies ahead. Many come to the office with absolutely no idea of what the job involves: they assume it is like being a glorified Secretary of State, rather than being the person who sets the direction for others. Too many incomers are not well enough equipped or willing to learn how to prepare to optimise their potential, with recent PMs in particular having little experience of running departments (a meagre three from Blair to Johnson). Only two of the last eight (Blair and Cameron) had the formative experience of having been Leader of the Opposition, which gives a unique if incomplete window on the breadth of the job.

Once inside the ill-configured Number 10, they fill it up with political appointees who know little about the way that government operates, but possess a belief, derived from scant evidence, that they, unlike their predecessors, will make the system work: as failure after failure among recent political advisors to the prime minister have shown, they don’t. Since May, the practice has grown up that the incoming prime minister appoints their own Principal Private Secretary, rather than inheriting the figure who served their predecessor who could tell them how the job and Whitehall operate. May, Johnson, Truss, and Sunak brought in the Principal Private Secretary from the Cabinet job they held before. The incoming prime ministers often have a vague and romantic notion of a golden era that never existed: the Cameron team wanted to get back to the small No. 10 of Thatcher, while Liz Truss wanted to recreate the No. 10 she recalled of Cameron.

The average prime minister, since 1945, has been in power on for four and a half years (compared to eight years seven months for the German Chancellor). This already short time is eaten up further when two or three years are spent learning about the job. Precious little time is left. The challenges and blows come in thick and furiously from day one. It makes holding to a steady course far harder for inexperienced prime ministers and hapless political aides. The job of Prime Minister is broader, quicker and more relentless than anything the incomer, whatever their background, will ever have experienced. The very pace makes some believe they always have to respond quickly, thereby disconcerting Cabinet and Parliament who feel bypassed. Prime ministers have struggled to turn social media to their advantage, while WhatsApp and other platforms have added to the ability of estranged colleagues to destabilise the PM, or of dissident MPs to organise themselves out of the sight of whips.

Peter Ricketts, Britain’s first National Security Advisor (2010–12), blames the pressures to react too quickly, and inability to carve out reflection time, for contributing to mistakes over the Iraq war, the EU referendum, and the failure to prepare better for COVID-19. Indeed, too often we have witnessed a series of knee-jerks rather than wise leadership and considered judgement. COVID-19 should indeed have been better anticipated by No. 10. Jeremy Heywood told me near the end of his life that lack of strategic planning at the centre was its great weakness. In 2019, historian Peter Frankopan gave a seminar at No. 10. His final words still float through its corridors: ‘my greatest worry for the future is a global pandemic’. The quality of the centre during COVID-19 was vastly inferior to its calm methodical work during much of the Second World War. What has been unlearnt?

In response to the core question of the book, and the title – have the undoubted challenges made the job of PM impossible? – the answer is, ‘no’. Agile incumbents throughout the 300 years have negotiated their way round the difficulties du jour, turned crises to their advantage, and come out on top. The job may have seemed to all fifty-seven at times to be impossible: but it is because of the way the incumbents have chosen to act in office; not because of any inherent unworkability of their office. Not in 1721: not in 2024.

The British prime minister, let us recall, is in an enviable position alongside comparable roles abroad: to repeat, not being pinned down or defined by a written constitution, not having to operate alongside a directly elected head of state, not operating in a federal structure that sucks power away from the capital (albeit happening now with devolved administrations), no longer tied by EU regulation or oversight, and not having an electoral system as on the Continent and beyond which often throws up coalition governments: the British electoral system, at least from 1945–2010, normally guaranteed a majority government. Not having the burden of a department to run means the prime minister can range more freely, while the refinements of the Cabinet Office and Number 10 play to their advantage, but only if they understand how to make it work for them. Technological innovations during the last century have worked greatly to their favour. Not in their dreams could Walpole, Pitt, Peel, or Gladstone have imagined talking directly to the nation, and with leaders abroad, at the push of a button. Poor PMs make excuses, most recently blaming ‘the blob’; the best PMs lead.

So if the job is not impossible, what changes could be introduced to give the incumbent a better prospect of making a significant impact in the office? We set out several below. The contiguous questions are: does the British political system produce candidates for the top job of the highest calibre who can maximise the job’s potential, and how can they be better prepared for what will greet them once in office?

How Can the Job Be Improved?

The prime minister is the most written about, but the most under-diagnosed element of the British constitution. More reporting and academic study has focused on the incumbents than on any other figure in Britain, the subjects of intense scrutiny from daily news, historical and political science treatises, and curricula at universities and schools. Biographies of individual prime ministers and edited books on prime ministers at large pour forth from publishers. What we have lacked are serious enquiries into the operation of the office of prime minister over time, and how it might perform better. So here are the proposals which the book has been cumulatively pointing towards to allow the office to perform better, and the country to be led better, as the PM enters its fourth century. There has been only one agenda-changing prime minister in the last seventy years: Thatcher. These changes might pave the way for more.

It’s the Finances, Stupid.

The biggest single factor responsible for the failure of the prime minister to achieve their ambitions has been their lack of control over finance, repeated clashes with the Chancellor, and lack of institutional resources since the PM gave up the Treasury in 1841.

Since the 1980s in particular, the Chancellor has accumulated a destabilising amount of autonomous power, with little or no constitutional justification or legitimacy for doing so. It has not helped that so many prime ministers – even former Chancellors, such as Churchill or Callaghan – are economically illiterate. But even when they have understood finance, as with Wilson or Brown, it can be even less guarantee of a smooth and productive relationship. The prime minister, not the Chancellor, is the nation’s chief executive, and for the system to work, the Chancellor has to be subordinate to them: if they don’t agree on fundamentals, the Chancellor has to go. Yet since Brown’s Chancellorship, they have (with the exception of Osborne and Hunt) regularly thwarted the will of the prime minister, for little reason other than that the Treasury has the personnel and gall, and the Chancellor the raw political power and knowledge, to do so. We have seen repeatedly that information is one of the PM’s greatest weapons: but finance is one area in which they do not monopolise it. The role of Chancellor has increasingly been seen as a ladder, and tool, in a Cursus Honorum leading to Number 10, and not, as it should be, a department that works with the prime minister.

How to reset the money dial is the question. One option would be to reduce the influence of the Treasury. But the numerous attempts to do so have failed, notably, the DEA from 1964–9. Chopping its power or breaking it up now, for example by creating an ‘Economics Ministry’, would be too disruptive and might work no better. It may work abroad, but it is alien to the British tradition. Putting a ‘mini Treasury’ within the Number 10 complex to beef up the PM’s information might to work better. A clear and widespread understanding is needed again that the job of Chancellor is to support the prime minister, and that it is the prime minister, not the Chancellor, who is ultimately the boss at the Treasury. The prime minister’s job, as it says, if not on the tin, then at least on the brass, i.e. the letterbox of Number 10, is to be the ‘First Lord of the Treasury’. The nation’s chief bean-counters, the Treasury, need to be reminded that first comes before second. Britain is governed significantly by convention. This particular convention has become clouded. It needs to be restored in full. To do so, the prime minister needs to assert their rights as First Lord.

Johnson’s establishment of a ‘joint economic unit’ in early 2020 never worked because it was underpowered. Truss’s economic unit failed because it only contained three special advisors, all of whom were appointed on ideological grounds, rather than some having Treasury/Whitehall experience.

Another option would be establishing a new body, the Economic Security Council (ESC), chaired by the PM, to mirror the NSC, bringing together all the key economic ministers and officials, to help redress the imbalance. Cameron is a keen supporter: ‘it is essential [if] the PM [is to be] in a commanding position’, he said. The Economic Security Council would require some overriding powers to break Number 10/11 feuds, perhaps with the ability to lock in specific areas of spending/cutting listed in a manifesto into a budget, preventing the Treasury’s ability to block a prime minister, similar to the House of Lords’ inability to stall fiscal bills in subordination to the Commons.

But we believe there are better ways to address this fundamental and long-standing problem for the PM.

Lack of Clarity of Mission.

Prime minister after prime minister arrives at No. 10 – in the last hundred years, MacDonald, Eden, Blair, Brown, and Johnson – with too little idea what they want to do in office, yet the job is unforgiving in allowing time for thinking once the front door swings shut. Our proposal is that, during the general election campaign and in the months leading up to it, or during the leadership campaign if a change mid-government, to be finalised in the first few days in office, the prime minister needs to draw up a detailed and costed ‘programme for government’. The Coalition’s five-year programme for change drawn up in May 2010 between the Conservatives and the Liberal Democrats, prepared in just a few days, shows what can be achieved. Every incoming PM should do this, drawing on the election manifesto, presenting costed programmes, anticipating trade-offs that will emerge, ensuring that Cabinet ministers, including and above all the Chancellor, are fully bound into a common programme of action.²⁷

Lack of Effective Institutional Resources.

From 1721 to 1841, the prime minister had the Treasury behind them, situated conveniently close at the end of Downing Street. The office lacked support after the loss of the Treasury, though some PMs like Palmerston and Salisbury drew on the resources of the Foreign Office. From 1916 onwards, the prime minister had the Cabinet Office to help them. But since the 1960s, the prime minister has increasingly struggled, not helped by pointless reorganisations and massive duplications at the centre, confusing them and everyone else across Whitehall. The Cabinet Office has lost its way – bloated, inadequately led, and incoherent, replicating work done elsewhere, while failing to ensure adequate oversight of areas not picked up elsewhere in Whitehall. Number 10 has been almost comically badly run with countless poorly performing new units, and with a near absence of coherence and a willingness to match office to PM function and need. Institutional memory has often been absent. Bodies that once worked well, like the fifty-year-old Policy Unit, splutter on regardless.

Rampaging into the vacuum at the centre charged the Treasury, and who can blame it? The PM was hopelessly outgunned, short on time, bandwidth, and resources. The Chancellor has come to have almost an open door to decide the government’s financial priorities in spending reviews.

In the place of this incoherent mayhem, we propose establishing a ‘Department of Prime Minister and Cabinet’ (PM&C) modelled loosely on the body that exists in Australia, which created a PM department in 1911, and the PM&C in 1971. This will allow the prime minister to reassert his authority that has been lost, principally to the Chancellor. Similar bodies are the Executive Office of the President in the United States, the Canadian Privy Council Office, and the Bundeskanzleramt serving the German Chancellor in Berlin. But the Australian example provides the best match for the UK, not least because it oversees the entire operation of the Civil Service, and because it has proven its quality and flexibility over fifty years serving different administrations and rising to fresh demands as they emerged. It ensures that the Australian Cabinet has detailed briefing on all issues to come before it, and it establishes a series of Task Forces to keep abreast of emerging issues. All senior officials across the Australian Civil Service spend part of their career working in the PM&C, helping make a cohesive system of government.

The British version of the PM&C, replacing the tired and failing No. 10 and Cabinet Office apparatus, should be organised into five separate divisions, serving the prime minister (physically situated in No. 10), overseeing Cabinet and Civil Service, then economics and finance, home and social, and finally foreign policy.

The prime minister should in this new plan initiate a dual leadership by officials. To oversee No. 10/Downing Street, there should be a Downing Street Permanent Secretary, and to head the new the Cabinet/Civil Service division there should be the PM&C Permanent Secretary. These two positions would replace the single and no longer effective post of Cabinet Secretary/Head of the Civil Service. Since the post of Cabinet Secretary was created in 1916, a series of remarkable long-serving incumbents held the post, but since the 1990s, the challenge of overseeing both jobs has been too much for any one person.

Lack of Space.

Space, like time discussed below, needs to be much better used at the centre. Valuable room is taken up in the cramped Number 10 by a variety of people and functions that could be carried out as well elsewhere in Westminster or Whitehall, freeing up space for senior officials and specialists in data, science, medicine, AI, and economics to provide the prime minister with top quantitative analysis, without having to rely on Whitehall departments to do so. It would mean Number 10 can interrogate far more thoroughly material coming in from across Whitehall, not least from the Treasury. Rupert Harrison is one of many key aides who worked in Number 10 who realised how under-powered it was, often when it was too late to do anything.

Number 10 doesn’t need to move to a brand-new building: but it does need to be peopled by more high-powered and knowledgeable staff, and be less full of in-and-out political aides who, however talented at party matters and winning elections, know little or too often nothing of their boss’s job of governing, and the environment in which he or she operates. Truss mistakenly stripped out Number 10 to a bare minimum, replacing most roles with aides rather than substantive minds experienced in governing. Some of the special advisers since 1997 would not be found anywhere near the chief executive of a large organisation. Why in Downing Street?

For far too much of its recent history, Number 10 has been chaotic, in a state of near constant flux, overseen by a prime minister who has little idea initially how to organise it, or by ‘chiefs of staff’ who have little understanding of the intricacies of delivering for their PM in Whitehall and Westminster. Number 10 has been relaunched and reinvented more often in the last forty years than pop superstar Madonna: and it seldom works, because there is negligible institutional memory or learning. This book shows repeatedly that prime ministers often performed best with a strategic, knowledgeable, and orderly Number 10. The prime minister, to repeat, is the head of strategy: tactics, operations, and delivery should be monitored and probed by Number 10 staff, freeing up the PM to range more widely.

If the PM oversees the first two divisions, who will the other three divisions in the PM&C (i.e. economics, home and foreign)?

Lack of Time.

The demands on the prime minister have grown vastly in the first three centuries, but the number of hours in the day has not. Prime ministers have played with the idea of having deputies (DPMs), and have often performed better when they have had one in either a formal (e.g. Major with Heseltine from 1995) or informal (e.g. Asquith) capacity.

We propose that the irregular DPM position should become formalised, with the three most senior Cabinet posts – Chancellor, Foreign Secretary, and Home Secretary – each stepping up and becoming DPMs overseeing the three remaining divisions of the PM&C and the Cabinet committees associated with them. The Chancellor would be responsible for the economic and financial division. The Home Secretary should oversee home and social policy. The Foreign Secretary should oversee the international division of the PM&C.

The offices of Home Secretary and Foreign Secretary established in 1782 are ripe for revitalising. The former position, chopped of many of its initial roles and powers, has long been known as the graveyard in British politics, with its own claim to the title of ‘impossible office’. It is ripe for revivification. So too is the forlorn post of Foreign Secretary. The leaching out of its work, with Britain’s dwindling power abroad and the prime minister filching many of the juiciest parts of the duties, means the Foreign Secretary has more time at their disposal than the seniority of the office would suggest. They should thus take over much of the routine external responsibilities from the prime minister and, when the occasion merits it, host meetings and receptions in Number 10 (because of its superior status over the more capacious Foreign Office).

The PM would have to feel secure enough to establish these three DPM posts: it has been the lack of trust at the very top that has proved often corrosive of good governance. Where prime ministers have trusted their deputies, de jure or de facto as Churchill did with Attlee in the war, Thatcher with Whitelaw, or Cameron did with Hague and Osborne, the deputy model worked successfully.

Prime ministers have for sixty and more years lacked thinking time, and for doing what they most need to do, being strategic, not distracted by tactics, operations, crises and day-to-day reaction. It has changed since Macmillan (who despite the nonchalant air he liked to exude was frequently overburdened, as he wrote in his memoirs). He even asked his former political adversary Attlee (who himself had said ‘I never felt under any sense of strain’ as PM) to examine whether the burden had become excessive by the early 1960s. No, the former Labour PM concluded, not entirely helpfully.²⁸ Use of the DPMs in this new model will allow the PM far more time than at present for what only they can do: ensuring that the principal policies and strategy of the government, on which it was elected, are being carried out; monitoring the performance properly (at last) of Secretaries of State running the departments; crisis management without crisis leadership; representing the country on the most important issues abroad; thinking long-term as custodian of the nation, and providing better oversight of the nation’s finances than the PM usually manages.

The prime minister will thus have more time too to meet a wider cross-section of people, engaging with them for more than just the current cursory conversations. More time for Parliament: their attendance has been in steady decline, paradoxically, since Britain became a full democracy in the early twentieth century. More time to consider national issues that transcend narrow sectional interests and the next general election. More time to go to the theatre and cinema, to art galleries, to read books again, and to use Chequers, as envisaged in its initial bequest: ‘[to] create and preserve a just sense of proportion’.²⁹ More time for their spouses, children, family, and friends which will ground and renew them. More time for exercise and their inner life.

The nation needs measured, not fraught, prime ministers, nor the physically and mentally unhealthy incumbents who have often fretted through its rooms over the last 300 years. More time to travel to all parts of the four nations for far more than just the rushed flag-waving trips of today. It is many years since the prime minister had regular overnight stays in Northern Ireland, Scotland, and Wales. They are prime ministers of the whole United Kingdom: fancy new titles, like Johnson’s ‘minister for the union’, don’t convince, but closer engagement might. Top rank leaders in all walks know how to pace themselves, to avoid overwork and burn out. Time is the prime minister’s most precious asset, and the current regimen is not allowing the incumbent to command or optimise it in the national interest.

Lack of Understanding.

The electorate might be shocked if they knew how little incoming prime ministers and their closest advisors know and understand about the history and operation of the office, its powers and constraints, and how the system works. Such ignorance would not be tolerated in an incoming leader of any other organisation, so why should we tolerate it in the British prime minister, the most important job in the land? They know less about how the system works than the above average Politics A-level student.

Prime ministers and their aides regularly trash their predecessors, which, aside from bad manners, shows an unwillingness to learn. Handovers are often perfunctory. Incomers then try to rewrite the rules of the operation, as if they were Walpole in year zero. Senior civil servants, who carry institutional memory between successor administrations, are now often binned on day one for being tainted by the old regime, or are outnumbered by political appointees in Number 10. They rarely last more than two years: too short to learn but too long in harm inflicted, none more damaging to the system of government than Johnson’s puppet master, Dominic Cummings. Lack of institutional memory causes avoidable errors. Philosopher John Gray highlights the difficulties: ‘If there is anything unique about the human animal it is that it has the ability to grow knowledge at an accelerating rate while being chronically incapable of learning from experience. Science and technology are cumulative, whereas ethics and politics deal with recurring dilemmas.’³⁰ New prime ministers arrive ego-heavy and history-light, with often the most rudimentary and jejeune notions that they want to govern/have a No. 10 ‘like Attlee’, ‘like Thatcher’, or ‘like Blair’ (‘didn’t he have a delivery unit?’), without any understanding of the context, reading up about, or reflecting on earlier regimes. Indeed, shockingly, prime ministers very rarely talk to their predecessors about their periods in Downing Street: what could they possibly learn?

One of the more startling facts in the book is that the last seven prime ministers up to Sunak served in only nine Whitehall departments between them before becoming PM, in contrast to the five prime ministers before them, from Heath to Major, who collectively had twenty-three jobs in Whitehall beforehand, and the five prime ministers before them, from Attlee to Douglas-Home, who served in thirty-nine ministerial roles. Experience of course is no guarantor of wisdom or competence: four of the nine Whitehall departments led by the last seven PMs were headed by Liz Truss: an ability to learn and to listen is essential.

So what to do about it? Ten years ago, I encouraged Cabinet Secretary Jeremy Heywood to set up the history group at Number 10 to try and instil more sense of collective understanding of past prime ministers, and how they and Number 10 have operated at their best. David Cameron was notably supportive. A parade of senior ex-Number 10 staff, biographers, and historians, like Andrew Roberts, have given lunchtime talks in the Pillared State Room. But the key figures rarely attended the talks, whether constitutional historian Vernon Bogdanor talking about the pitfalls of Wilson’s 1975 referendum on the EU, nor indeed Peter Frankopan. Efforts to enshrine history and institutional memory – not to be mistaken for inertia – have stalled.

The prime minister should appoint a Chief Historian at the centre with the same status as the Chief Scientific Advisor, with historians equally in every Whitehall department, not to become buried in archive preservation, but to compel ministers to reflect on previous experiences, and to learn from the past. Their job is to intertwine historical methodology and memory into the decision-making and policy process. Remembering what worked, and more importantly what did not, will save prime ministers and Cabinet time, money, political capital, and the embarrassment of forgetting their own history when the inevitable U-turns come over a historically illiterate policy or decision.

Prime ministers need to have a much clearer understanding of what works, and what doesn’t. The Institute for Government, set up in 2008, has done important work in spreading understanding about what ministers need to do to govern more effectively.³¹ Its ‘Commission on the Centre’ report in early 2024 lays out a better future for the PM, Cabinet Office, and Treasury. Much could be learnt too from staff colleges in the military, not least about how the prime minister should deport him or herself.

The separate functions in Number 10, Whitehall liaison, policy, communications, foreign affairs, parliamentary, travel/logistical, etc., need to be more regularised, as they are in the offices of national leaders abroad, so that, when each new prime minister arrives, they have in place highly proficient staff who show them how the particular area operates and how they can make it best work for them. Number 10 needs to move from being a chronically amateur into a professional and sleek outfit. Strong leaders everywhere encourage diverse points of view around them: yet far too many prime ministers want their ‘mates’ to come into Downing Street, and are then surprised about ‘groupthink’. Diversity in every way, including social background, regions, gender, and ethnicity, needs to be deeply embedded at the centre of government. Government needs to learn more from abroad too: ‘it almost never looks to other countries to see what could be done better’, laments former policy chief at Number 10, Camilla Cavendish.³²

Britain does not need to have a written constitution. It does not need to have still more know-nothing special advisors. It does not need electoral reform. It does need changes in the areas listed above to be implemented, and then to settle down.

Was It All Worth It?

Our journey through history is all but over. We conclude with a meditation on the job itself. We need better quality candidates to present for premiership. Why would they want to today? Prime ministers are human beings, who bleed and hurt and suffer. Throughout the book, we have accentuated the human nature of the job. Politics is a harsh game. MPs aspire to be ministers, ministers to be promoted to one of the top four posts, and most of those want to become prime minister. Very few manage it. No one makes them do it, but those who do scale to the top of what Disraeli described as the ‘greasy pole’ are often far from happy with the experience and their legacy. No PM in the last hundred years has left at a moment entirely of their own choosing.

The job should carry a health warning. Seven have died in office, and five dead within a year of leaving, with a further three within three years. Within ten years, half the fifty-seven PMs were dead. Given how young many were on arrival, it’s not a great prospect. Remarkably few achieve what they hoped. Most leave involuntarily. In office, they are criticised, mocked, and undermined relentlessly. The media are merciless. The deranged and terrorists constantly want to kill or kidnap them and their loved ones. There is no peace. Many experienced pain earlier in their lives: one study suggests two thirds in office between 1812 and 1940 lost a parent in childhood, and asks whether their quest for power and prestige was motivated by protection against emptiness and insignificance.³³ This is dangerous territory, but there can be no doubting that only driven personalities want to become prime minister, and that the nervous strain of office on often outlier personalities is considerable.

If it is not that good a time for the prime minister, it is worse for their families. The experience of having a parent who is prime minister can put almost unbearable strain on children. The lack of normality in their lives, the difficulty of being seen only for who their parents are rather than who they are, the burden of expectation that goes with that, and parental attention often sacrificed for the political career causing a degree of neglect, all take their toll. Inherited genes, which might have powered the parent to the top, can play out less well for their children. Yes, some have escaped the shadow and have had normal lives. But to take just one period of twenty years, three of Churchill’s children died unhappily, Diana at the age of fifty-four, Randolph at the age of fifty-seven, and Sarah at the age of sixty-seven (Marigold had died at the age of three). Addiction or mental illness afflicted all three of them, and it was only Mary, the youngest, who had a long and stable life, marrying Christopher Soames, one-time British ambassador to Paris, and dying at the age of ninety-one. Eden’s elder son Simon was killed in action at the end of the war in June 1945, while his younger, Nicholas, died aged fifty-four from complications from AIDS. Macmillan’s marriage was fraught due to the prolonged affair between his wife Dorothy and the louche Conservative politician Bob Boothby. Their daughter Sarah died aged forty, suffering from alcoholism, an illness that also afflicted his son Maurice, who had a brief and not very successful political career under Douglas-Home.

Being a prime minister offers no protection from the agonies that can afflict all parents. Asquith’s son Raymond was killed in action in 1916, in a war which saw five of Salisbury’s ten grandsons dying, Rosebery’s son, and two of Bonar Law’s sons. ‘Night seems to have descended on him … he could only sit despondently gazing into vacancy … obliterating light and happiness’, wrote Bonar Law’s biographer Blake of the losses.³⁴ Gordon and Sarah Brown’s daughter Jennifer died in January 2002 soon after her birth, before he became PM. David and Samantha Cameron’s son Ivan died aged six in February 2009. Rarely has a more moving House of Commons speech been made by a PM than by Brown when he offered the Camerons his condolences for ‘an unbearable sorrow that no parent should have to endure’.³⁵

Few spouses opted to be married to a prime minister, and while some managed to enjoy the experience, as many were unenthusiastic. Their own careers were dented, their lives put on hold, while living in the flat at the top of Downing Street offers no privacy or escape. Only in the last twenty-five years have they been paid by the state for what in effect has become ‘a job’. Since 1945, all except the bachelor Heath took their spouses through the indignity of a very public departure: Attlee unceremoniously dumped in 1951 despite having won more votes than the Conservatives, Churchill finally eased out by his Cabinet, Eden in disgrace after Suez, Macmillan his government’s focus lost and believing he was more unwell than he proved, and Douglas-Home ejected by the electorate. Wilson, cited as the exception who left at a moment of his own choosing, was a shadow of his former self in his final two years, suffering from alcohol and memory loss. Heath, Major, and Brown were all ousted in general elections, while Thatcher, Blair to some extent, and May had lost the confidence of their colleagues. Cameron resigned after the catastrophic policy reversal of defeat in the EU referendum which he had called. We might contrast the PM’s first euphoric words with their final ones departing Number 10. The tears they shed do not always fade in the empty years ahead.

Prime ministers are often at a loss to know what to do after they resign. North, Portland, Goderich, Addington, Wellington, and Russell all came back into government. In the twentieth century, Balfour and Douglas-Home returned as Foreign Secretaries, while Chamberlain continued as a minister in Churchill’s wartime Cabinet as Lord President until too ill. In the twenty first century, David Cameron returned as Foreign Secretary in November 2023, fifty three years since the last prime minister to return to Cabinet, much to the surprise and shock of the Westminster. Handling ‘post-premierships’ can be tricky when there is no equivalent status to that enjoyed by former US or French presidents: some money from the state for office staff, and lifelong police protection, are scant consolations. Finding a job in the private sector can be awkward, finding a role in government at home or abroad difficult, and a return to politics now ruled out. If their successors fail, they can feel frustrated; worse, if they succeed, they can feel inadequate, while all the time they watch on as their former colleagues and friends diminish what they tried to build. Heath retreated to his house, which he lavishly decorated with memorabilia, in the Cathedral Close in Salisbury, but remained deeply scarred by the way he believed he had been treated by Thatcher. Thatcher herself, angry at the way she was ousted, and by the direction in which Major took the party, had as melancholic and unsatisfactory a post-premiership as any since Eden.³⁶

Major in contrast has had a sunny post-premiership, one he has certainly enjoyed more than his seven years in Number 10, with a status and respect as an elder statesman he never enjoyed inside. No former prime minister has tried harder to build an independent career than Tony Blair, setting up his own foundation, and working for progress in the Middle East and Africa, with climate change and inter-faith dialogue. But he has not had a sympathetic press, which never forgave him for the Iraq War, while he has learnt the bitter truth that world leaders are not very interested in former prime ministers. Brown, and May have all chosen to make periodic political interventions, and all have many years ahead of them to do so. Indeed, there are now as many post-prime ministers alive, seven, as at any point in history. The year 1842 saw four living former prime ministers, Goderich, Grey, Wellington, and Melbourne. In 1985, five former prime ministers (Macmillan, Home, Wilson, Heath and Callaghan with four of them still in either the Lords or Commons with the exception of Wilson). In December 1985, Thatcher invited five former prime ministers to No. 10 to dinner with the queen. It’s a lot of talent for the government, short of experience and wisdom, to squander.

In Cameron’s final days in office, their children, for a treat, saw a one-off show by the Royal Shakespeare Company, performed in the walled garden at the back of Number 10. Extracts from the Bard’s plays were acted out, selected before his rushed departure was known. None foresaw the poignancy of the Cameron family, huddled together in the front row, watching the murder scene from Julius Caesar, and Lady Macbeth plotting the death of Duncan. A few days later, they left Number 10 for good. That evening, daughter Florence turned at bedtime to her father and asked, ‘Daddy, when are we going back home?’³⁷

Cameron, for the first time since Alec Douglas-Home in 1970, returned to Cabinet in 2023. Such a political afterlife will likely, if sadly, remain a rarity. Was it all worth it, for the former prime ministers, their spouses, and children? They would of course say yes; but deep inside, they must wonder, as they have struggled after Downing Street to find a home, come to terms with the unfulfilled dreams for their premiership and reclaim their disordered lives.

Book contents

Chapter 10 - The Impossible Office?

Summary

Keywords

Premierships are Not Pop Songs or Films

Tier I: Agenda Changers.

Tier II: Major Contributors.

Tier III: Positive Stabilisers.

Tier IV: Noble Failures.

Tier V: Ignoble Failures.

Tier VI: Left on the Starting Line.

The Overall List

So What Makes for a Successful Premiership?

Can the Verdict on a Premiership Be Improved?

Has the Job Become Impossible?

How Can the Job Be Improved?

It’s the Finances, Stupid.

Lack of Clarity of Mission.

Lack of Effective Institutional Resources.

Lack of Space.

Lack of Time.

Lack of Understanding.

Was It All Worth It?

Book contents

Chapter 10 - The Impossible Office?

Summary

Keywords

Premierships are Not Pop Songs or Films

Tier I: Agenda Changers.

Tier II: Major Contributors.

Tier III: Positive Stabilisers.

Tier IV: Noble Failures.

Tier V: Ignoble Failures.

Tier VI: Left on the Starting Line.

The Overall List

So What Makes for a Successful Premiership?

Can the Verdict on a Premiership Be Improved?

Has the Job Become Impossible?

How Can the Job Be Improved?

It’s the Finances, Stupid.

Lack of Clarity of Mission.

Lack of Effective Institutional Resources.

Lack of Space.

Lack of Time.

Lack of Understanding.

Was It All Worth It?

Save book to Kindle

Save book to Dropbox

Save book to Google Drive