Mr Abbott prompted an outcry in question time on Thursday when he compared Mr Shorten to the Nazi Joseph Goebbels, just over a month after he accused the Labor Party of causing "a holocaust of job losses".

Joseph Goebbels was the minister for propaganda in Nazi Germany and one of Adolf Hitler's closest associates.

Shadow attorney-general Mark Dreyfus, who is Jewish, was booted from the chamber after he leapt to his feet to protest the remarks. There was outcry from Labor with at least four opposition frontbenchers standing up in protest and a Jewish Labor MP Michael Danby walked out in protest.

Mr Abbott was responding to a question from Mr Shorten about whether a debt level of about 60 per cent of GDP would "see Australia lose its AAA credit rating".

"Such a question from such a Leader of the Opposition, it's like the arsonist complaining about the fire," Mr Abbott said.

He continued to describe Mr Shorten as "the Dr Goebbels of economic policy" for his promotion of Labor's economic management in government before immediately crying "I withdraw, I withdraw, I withdraw, Madam Speaker".

Speaker Bronwyn Bishop was asked to reconsider removing Mr Dreyfus but refused.

Leader of the House Christopher Pyne pointed out that Mr Dreyfus had used "exactly the same description" to describe Mr Abbott in opposition.

In a 2011 opinion piece, Mr Dreyfus said there was a "Goebbellian cynicism" to Mr Abbott describing his campaign on the carbon tax as a truth campaign.

"Abbott's wildest claim is that he is running a 'truth campaign'," he wrote. "Leaving aside the Goebbellian cynicism of labelling a scare campaign a 'truth campaign', I think it shows Abbott's contempt for the Australian electorate."

In question time, Mr Abbott said: "I do withdraw and I do apologise for using that phrase.

"But, Madam Speaker, the truth is the Leader of the Opposition was claiming to his people that the government in which he served had delivered a surplus."

Follow us on Twitter