Book review: The Pivot of Power - how each prime minister shaped the bureaucracy

From the first prime minister Edmund Barton's inspired appointment of Robert Garran as the first public servant, the hand of each incumbent left its mark. The growing complexity of government and public administration is in itself a fascinating sub-narrative in Australia's history; it has been a long journey from the time Garran, who once said he carried the whole business of the Commonwealth in his briefcase, could write: "I was not only the head [of the department], but the tail. I was my own clerk and messenger. My first duty was to write out with my own hand Commonwealth Gazette No. 1 proclaiming the establishment of the Commonwealth and the appointment of ministers of state, and to send myself down with it to the government printer."

The legendary Garran is the professional public servant's historical role model, and his ability to serve prime ministers as radically different as the courtly Alfred Deakin and the irascible Billy Hughes, while responding to their demands, neatly encapsulates the character of that critical relationship between the head of government and those responsible for advising on and implementing policy.

Just how prime ministers attain power is always interesting, with ambition, skill, luck, ruthlessness and occasionally bastardry all coming into play. And how they lose it is invariably the stuff of drama, high and low. But in between winning office and vacating it (almost always unwillingly), they need to perform a role and, while what they do and how they do it lacks the dramatic impact of coming and going, it is something history must know if we are to make any sense of the political process and those who drive it.

This is the focus of a book by three academics (The Pivot of Power: Australian Prime Ministers and Political Leadership, 1949-2016, by Paul Strangio, Paul 't Hart and James Walter), the second volume of their meticulous study of the Australian prime ministership and its evolution. They look at how each prime minister operated and the legacy each has left in shaping the office and the machinery of government in general.

The prime minister has increasingly become the focus of government but, as the authors remind us, prime ministerial leadership is seldom if ever the outcome of one individual, however gifted, acting alone. It is about essential collaborative relationships formed at the centre of power; those webs of influence, formal and informal, that radiate out from the prime minister's desk.

The challenges are as daunting as they are immense: promises need to be kept and sometimes broken; an electorate needs to be carried; a party's confidence must be maintained; ambitious or hostile colleagues must be managed; parliament needs to be handled, and if possible, dominated; an opposition needs to be bested – and all these things before actually running a country. No wonder stumbles and falls are not at all uncommon.

One of several metrics the authors employ is analysing the effect each prime minister had on the public service. By carefully examining the changes each wrought to best serve his or her needs, we see an evolutionary process unfold as new circumstances arise and new and complex problems emerge to be addressed.

This second volume opens in 1949 with the coming to office of Robert Menzies, the man who returned from the political dead, founded a new party and dominated the political stage for a generation. In treating Menzies, the authors offer what is arguably the most finely nuanced account to date of this still enigmatic figure, who had the ruthlessness and political gifts to not only resurrect himself from ignominy and humiliation, but survive at the top for so long, thanks to guile as well as luck, and also some remarkable public servants whose talents Menzies astutely employed. While the authors are critical of many of his actions in his record 16 years in the job, they do not hesitate to give him unstinting credit for his often overlooked role in developing the Australian Public Service's professionalism.

The postwar economic boom was something of a golden age for Australia, and the authors characterise the Menzies era as one of "managed prosperity". Menzies showed great faith in the public sector in giving it a key role in shaping society, most notably in devising and implementing policies that enabled advances in relative equality and social integration; mediation of the balance of advantage between different economic groups; and the near universalisation of educational access and standards.

Some bad economic weather early in the Menzies term, brought about by galloping inflation and harsh measures to rein it in, sparked tensions between Menzies' political objectives and his reliance on Treasury advice. (Menzies spoke in private of "Treasury witchcraft" and lamented its failure to understand public relations).

Nevertheless, his public articulation of responsible economic policy closely followed the Treasury brief, whatever his private misgivings. Policy inconsistency, however, led to conflict within the government and the higher echelons of the public service, as when Menzies sided with the formidable Roland Wilson, the Treasury secretary, in Wilson's periodic stoushes with trade minister John McEwen and the equally formidable Jack Crawford, the Trade Department secretary, and also with the influential H. C. "Nugget" Coombs, successively governor of the Commonwealth Bank and the Reserve Bank, over who should exercise primary authority in providing economic advice.

The escalating tensions reached a peak in 1965, the final year of Menzies' tenure. On Wilson's advice, Menzies impugned the long-awaited final report of the committee of economic inquiry, chaired by James Vernon, which the government had set up after the disastrous "credit squeeze" of 1961. The Vernon report's main thrust was to recommend an advisory council on economic growth, but Wilson would not hear of an alternative source of advice that might challenge the Treasury, and Menzies killed the proposal.

Under Menzies, the public service had grown in stature and influence; Menzies showed trust and confidence in his powerful mandarins. While Menzies built on the service's institutional development under his two Labor predecessors, John Curtin and Ben Chifley, his greatest achievement, scarcely matched by any of his successors, was to establish effective relationships between the bureaucracy and the executive. This had the effect, the authors write, of giving the so-called "seven dwarfs" (notable bureaucratic chiefs of great intellect but short stature) the scope to drive the service's professionalisation.

Menzies, to his credit, resisted pressure from his party to sideline officials who had worked with the previous Labor administrations. This trust was repaid in the relationships he forged, with the emphasis on integrity and professionalism. Political scientist Alan Davies observed that Menzies' approach saw a "relatively free hand given to bureaucrats in running their enterprises", which resulted in "a high level of bureaucratic innovation". John Bunting, sometime head of the Prime Minister's Department, called it "an integrated enterprise ... between the prime minister (and ministers) and public servants".

Menzies was succeeded by Harold Holt, who had little time for innovation before his untimely death at the end of 1967. But his tenure was notable for setting up the Australian Council for the Arts, on the advice of, and initially under the direction of, Coombs.

Holt's successor, the unorthodox John Gorton, who, like Holt, struggled for traction in the post-Menzies era, inherited a fractious party and some determined enemies. But in his administrative arrangements – most notably the appointment of the formidable Lennox Hewitt to run his department – Gorton was responsible for creating a more assertive policy role for public servants that subsequent administration accepted and adopted.

William McMahon, who toppled Gorton in 1971, left little in the way of lasting achievement. He was, the authors write, "the manipulative schemer who did whatever it took to get to the top and, once there, did not know what to do with the prize".

At the end of 1972, McMahon was defeated by a resurgent Labor Party under Gough Whitlam. Its innovative "It's Time" campaign inaugurated a new era for the country and, especially, public administration.

If the public service had grown in professionalism and influence in the postwar years, Whitlam took administration in a new direction. While the APS remained a key source of policy advice, that advice was now contestable; it became the catalyst for "a fundamental amplification of executive power". The aim was to bring advisory networks, developed and cultivated in opposition, into government; multiple sources of advice (so went the theory) would ensure robust debate.

Under the "leader-centric" Whitlam, the prime minister's office assumed a vastly expanded role of managing government priorities, coordination across portfolios and the difficult task of separating political demands from policy objectives. But the office was too small for such a complex brief, and much of the work was eventually shifted to an expanded Department of the Prime Minister and Cabinet.

However, while initially overambitious, the network approach that Whitlam pioneered, through reaching out into the party, academia, the media and the community in a way that was never open to the APS, remained an important function of the prime minister's office, and was employed to varying degrees by his successors.

The activist role of government that Whitlam championed created difficulties as much as opportunities, and the government was not long in becoming embroiled in energy-sapping disputes with the states, especially over the role of the Treasury and its concerns about spending. It soon became "government by crisis", the authors trace the government's decline to policy overreach, poor judgment and policy incoherence.

Whitlam's "leader-centric" approach was assumed by his successor, Malcolm Fraser, who also boosted the resources of his private office and department to widen the range of policy options. His leadership style was characterised by intense application, immense demands on his staff, advice from personal networks and an unrelenting workload on his ministers. Fraser, in time, was also accused of running a government by crisis, on which he appeared to thrive.

Bob Hawke, who succeeded Fraser in 1983, and Paul Keating, who toppled Hawke in 1991, are remembered for radically restructuring the Australian economy and rejecting shibboleth. Hawke, whom Keating served as treasurer, brought managerial flair to the prime ministership, operating "as the spider in a carefully constructed web of core executive arrangements". While avowedly hands-off, Hawke's imprimatur – or at the least his acquiescence – was needed for every major initiative. Like Fraser before him, Hawke was committed to strengthening the prime minister's institutional resources and demanded public service responsiveness to his government's views. Hawke, aware of the chaos of the Whitlam years, insisted on strict cabinet solidarity.

Keating, who was determined to put his own stamp on the job he wrested from Hawke, had a style all his own, forging exceptionally close relationships with senior officials, particularly Treasury secretary Bernie Fraser (subsequently appointed to head the Reserve Bank) and Fraser's successor at the Treasury, Ted Evans, both of whom shared Keating's world view and became loyal warriors in the intense policy debates of the 1990s.

Out of favour with Keating was Mike Codd, head of PM&C under Hawke, whom Keating saw as too supportive of Hawke's "cooperative federalism", which Keating rejected as too soft on the states. He replaced Codd with Mike Keating, with whom he built a close working relationship despite PM&C losing some heft, with the new prime minister discouraging it from dictating to other departments.

John Howard defeated Keating in 1996, and the public service braced for major change. Building on his predecessors' work, Howard further consolidated prime ministerial government by developing the executive and public service resources available to him. His command of the communications effort controlled debate and maintained discipline, in the process "developing a machine of unparalleled influence".

Howard was intent on changing the service's institutional culture. He turned to Max "the Axe" Moore-Wilton to implement the revolution. Moore-Wilton warned staff at PM&C to prepare for "presidential style government"; he was none too subtle in intimating that serious engagement with the Howard government required being onside with its objectives. Critics saw this as abandoning the convention of the service giving considered advice.

Moore-Wilton relinquished his post in 2003 and was succeeded by Peter Shergold, whose was far less confrontational and who took great pains in spelling out the rationale for Howard's public sector reforms. However, the authors note that, despite their different styles, the project of Moore-Wilton and Shergold was "integrated governance", with attention to Howard's priorities, strategic central-agency coordination and monitoring of implementation and delivery – all under the purview of PM&C.

Despite Howard's commitment to cultural change in the APS, he was seen as good to work with – invariably on top of submissions and papers, administratively efficient, calm, courteous and measured. Yet, the authors say, there was an undeniable edge: Howard's tenacity translated into tackling apparent divergence from the government line with punitive vigour, seen by some as intentional intimidation.

Howard's Labor successor, Kevin Rudd, established his public image in opposition, largely on a morning television talk show. But after his successful "Kevin07" campaign in 2007, he came to believe his power derived from the public, not caucus or cabinet. Coupled with his propensity to micromanage, it proved too much for his party, which revolted and dumped him. Rudd was undone by a series of dysfunctional relationships: within his own office; between the office, the wider government and bureaucracy; between Rudd and the party, cabinet and, fatally, caucus. (One senior bureaucrat told me at the time: "We all liked Kevin 'till we had to work with him.")

His successor, Julia Gillard, had policy successes, but failed to communicate them to the electorate. A collective sigh of relief in Canberra at Rudd's exit saw the head of PM&C, Terry Moran, seize the opportunity to reboot ruptured relations between the prime minister's office and PM&C and the public service more widely. Rudd returned briefly, lost the 2013 election and was succeeded by Tony Abbott, who also saw the merits in a powerful private office. But Abbott's influential chief of staff, Peta Credlin, eventually proved to be a major factor in the party turning against him.

Distrust coloured Abbott and Credlin's approach to the public service; three departmental heads were dismissed while another was given notice – all on the perception of having been too close to the former Labor government.

Prime ministers of late have had a hard time sustaining their parties' support; Rudd (2010), Gillard (2013) and Abbott (2015) were all ousted by their own side. The paradox the authors identify is that the increased power and influence prime ministers have accumulated has made them even more vulnerable, reinforcing leader centrality and augmenting public expectations.

As a tract on the increasingly complex field of public administration, this study highlights what has become an ever more perilous path to tread of simultaneously holding onto power and somehow directing the affairs of a nation. It has become, in some ways, an impossible job.

Dr Norman Abjorensen, of the Australian National University's Crawford school of public policy, is currently a visiting professor at Ateneo de Manila University's Ateneo school of government in the Philippines. He is the author of The Manner of Their Going: Prime Ministerial Exits from Lyne to Abbott (2015). norman.abjorensen@anu.edu.au

The Pivot of Power: Australian Prime Ministers and Political Leadership, 1949-2016, by Paul Strangio, Paul 't Hart, James Walter, is published by Miegunyah Press (August 2017).