Federal minister Greg Combet says Australians want politicians to stop the name-calling and get on with the job of government.
His comments follow a tumultuous week in federal politics in which Prime Minister Julia Gillard branded Opposition Leader Tony Abbott sexist and a misogynist.
Ms Gillard was in turn called a hypocrite for backing former Speaker Peter Slipper after the opposition demanded he be sacked for sending offensive text messages referring to female genitalia.
"It's been a pretty robust week alright," Mr Combet told ABC television on Sunday.
"But I think it's important for us to get back onto the public policy issues that people are more interested in."
People outside politics would "prefer politicians to get off name-calling and get on with the business of government", he said.
Asked whether he thought Mr Abbott was a misogynist, Mr Combet said: "He's a very aggressive, arrogant sort of fellow and he likes to lead a lynch mob."
Opposition frontbencher Sophie Mirabella said the government had "totally over-egged" the sexism tag.
"Julia Gillard and the whole Labor Party voted to defend the indefensible," she told Sky News on Sunday.
"Even Peter Slipper knew his position was untenable and Julia Gillard will wear that hypocrisy around her neck."
Meanwhile, Australian Greens leader Christine Milne accused Labor senator David Feeney of sexism.
Senator Feeney recently tweeted a series of pictures that feature the same image of Senator Milne with the caption, "the different emotional states of Senator Christine Milne".
Senator Milne said there was a "huge sexist element" in the tweets.
"If you're going to call it out, you have to call out sexism on all sides, regardless of who's responsible and that includes David Feeney for the government," she said.