Senator Abetz, who oversaw the public service under former prime minister Tony Abbott, says cutting the bureaucracy would show leadership.

The former employment minister used a column in his local Hobart newspaper on Thursday to criticise the public service's growth since he was in charge.

"When I was responsible for the public service in 2013, I had the media and everyone within government telling me we couldn't trim the size of the public service by the numbers which we set out in our policy – to be achieved through natural attrition and discipline," he wrote in The Mercury.

"I met with the Public Service Commission weekly and personally kept on top of the policy to ensure no breakouts and, within two years, we achieved a 14,000 reduction in the size of the public service, resulting in an annual $1.4 billion saving ...

"Regrettably, in just one year, since we've seen almost 4000 more staff added to the Commonwealth payroll."

As minister, the senator implemented a government-wide "recruitment freeze" after the Coalition won the 2013 election, which prevented agencies from hiring staff unless there were exceptional circumstances.

By the time he lifted the freeze in 2015, he had overseen the loss of about 15,000 Australian Public Service jobs.

However, the government was unable to shed staff by natural attrition alone: the number of redundancies during that period was about triple the usual rate.

Senator Abetz is a powerful influence among conservative Liberal parliamentarians, though he no longer holds any government roles. His views regularly mirror those of ousted leader Mr Abbott.

The senator argued on Thursday that the cost of the extra staff employed under the Turnbull government added about $400 million a year to public spending.

"For me, that's a pretty simple proposition. More public servants in Canberra or more support for small and medium-sized businesses right around Australia creating jobs ..."

He told ABC radio on Thursday morning that Liberals should "always be on the lookout to reduce the size of government and the size of expenditure".

"One place where I believe the government could provide some leadership is in the reduction of the public service."

Most of the new public service jobs created since Malcolm Turnbull became Prime Minister in September 2015 are non-ongoing, contract roles.

In an interview with Fairfax Media on Thursday, Senator Abetz acknowledged that shedding public service jobs was not always enough to reduce spending.

Government spending on consultants increased sharply after the Coalition's 2013 election win, while the bureaucracy suffered steep staffing losses.

The senator said this showed the government "needs to be disciplined, because public spending is always courtesy of the Australian taxpayer".

"And if you're just shifting the expenditure from one sector to the other, the burden for the Australian taxpayer remains the same and so not much is actually achieved."

Nonetheless, he said that "in principle, I would prefer the private sector to be providing [public] services".

"But clearly there are some areas where it is important to have public officials, and I suppose the most glaring example of that would be a police force," he said.

"But, at the end of the day, it is the quality of the service provided and the cost which, quite rightly, excites the interest of the Australian taxpayer and that is what the Australian government at all times has to keep its eye on ...

"I argue for the best possible quality and service, and reduction in cost, and whether that is then provided by the public service or the private sector is, from my point of view, basically a moot point."

Industry Super Australia chief economist Stephen Anthony said while there was a structural budget deficit, the answer was both tax reform and spending constraint.

The best way to approach spending reductions was to consider who may benefit from a strategic and comprehensive reform of tax and expenditure, and impose the burden of adjustment on those people.

The starting point in considering spending was identifying the priorities of government and ensuring services were being delivered in the most efficient and effective way.

The approach to public service cuts suggested by Senator Abetz risked taking talent and institutional memory from departments, Dr Anthony said.

"You often find the best people leave when you offer voluntary redundancies. You don't want to light a fire under their feet."

The Canberra Business Chamber said the ACT economy needed to continue diversifying to weather the impact of changes in the size of the public service.

The ACT had to ensure that public servant skills were transferable for positions in the private sector, which needed to keep growing to absorb government staff made redundant, chief executive Robyn Hendry said.

"We don't need to read a column in the newspaper to understand the position we need to be in from time to time to cope with the changes."

While the chamber wouldn't welcome "wholesale reductions" in public service jobs, there was evidence during the period of cuts in 2013 that government staff took private positions in Canberra instead of leaving the city, she said.

Senator Abetz suggested the cuts could free spending to help small businesses, however Ms Hendry said that reductions didn't immediately help them.

Although governments needed to identify efficiencies, targeted and specific reduction programs reduced the impact on the economy.

"As soon as you threaten jobs in some way, then of course that dampens consumer confidence."

Given Senator Abetz was a backbencher, she said his comments wouldn't have a large impact.

"It appears it's a philosophical approach to small government he's announcing, and that might be unsurprising to many."

Further public service cuts would increase the number and severity of problems facing the government, CPSU national secretary Nadine Flood said.

"Senator Abetz is conveniently ignoring the disastrous impact of his public sector job cuts on essential public services. As a direct result this government has lurched from crisis to crisis: the Centrelink robo-debt crisis and census debacle are just two examples.

"This Canberra-bashing from Senator Abetz is hardly surprising but the reality is the vast majority of Commonwealth public sector jobs are based outside of the ACT, and the thousands of job cuts under the Coalition have damaged regional communities and economies in Tasmania and around the country.

"That has clearly hurt rather than helped small businesses."

Labor MP for Fenner Andrew Leigh said public service cuts would mean longer wait times at Centrelink, poorer border security and inferior policymaking.