Scott Morrison has refused to declare confidence in the corporate watchdog or its chairman as expense scandals are investigated.
Mr Morrison also distanced himself from the appointment of James Shipton during his time as treasurer, saying other cabinet ministers were also involved.
The prime minister would not be drawn on whether a broad restructure of the Australian Securities and Investments Commission was needed, as suggested by some government members and business leaders.
"There is an investigative process under way ... and I don't think it's helpful for me to be offering commentary about that while that's under way," he told reporters in Canberra on Wednesday.
"I don't think that's fair to those who are involved."
The prime minister said Mr Shipton was right to stand aside while the investigation took place, and he would wait for the recommendations to be released before taking any action required.
"That's how you deal with those problems, that's how I will always deal with these problems, that's how you run a government," Mr Morrison said.
Labor has demanded the investigation be wrapped up within two weeks.
Mr Shipton is on paid leave after standing down when it was revealed taxpayers paid more than $118,000 for his personal tax advice.
Shadow assistant treasurer Stephen Jones said the scandal was extraordinary.
"There is no disagreement over the facts here. It seems pretty clear what's gone on - payments were made, they were unauthorised and unlawful," he told AAP.
"The only question to my mind is who knew? Did the prime minister know? Did the treasurer know?"
ASIC deputy chair Daniel Crennan quit after claiming almost $70,000 from taxpayers to cover rent.
Vivienne Thom is investigating both issues, but has until the end of the year to report her findings.
Mr Jones believes the review should be completed within two weeks.
"It's a devastating blow to ASIC," said.
"At a time when we need the corporate cop to be on the top of its game, it's actually on the bottom of the ladder."
ASIC acting chair Karen Chester told a Senate estimates hearing the review would look at whether processes, governance or something else were to blame.
"There's been failings," she told the committee.
"You have to go through and do a root cause analysis to understand what caused those failings."
Australian Associated Press