Protests this week on the steps of parliament have included threats to kill the premier after a doll depicting Daniel Andrews was thrown at a makeshift wooden gallows on Monday.

A protester was applauded by the crowd on Tuesday after she called for politicians to "dance on the end of a rope".

Despite some threats being posted to social media, Victoria Police are yet to arrest anyone involved.

Police did speak to the driver of a black Toyota Landcruiser, which was towing the gallows, and issued the 48-year-old Badger Creek man a warning related to traffic offences.

But Victoria Police say they are investigating "a number of recent incidents involving threats directed at members of parliament".

"As those investigations are ongoing and relate to security matters, it would be inappropriate to provide further details at this time," a Victoria Police spokeswoman said.

Several people had been charged over the past year for "threats directed at high profile members of the community, including members of parliament".

"Victoria Police takes any threat against a person extremely seriously, including both online and physical threats, and will investigate incidents to the fullest extent to hold offenders to account," the spokeswoman said.

Alleged far-right extremist Imre Pelyva, 36, has been charged with incitement-related offences by the counter-terrorism unit and will face court next week, AAP has confirmed.

Politicians including the Victorian premier and the state's opposition leader have expressed their concern about the protests on Thursday morning.

It comes after Liberal MPs David Davis, Bernie Finn, Roma Britnell and Craig Ondarchie all attended the protests earlier this week.

Mr Andrews labelled the protesters "extremists" and accused other politicians of playing "political games about lockdowns".

"People can threaten me all they like, they can threaten my wife and my kids all they like. It's the safety of every family that gets me up every morning," he told reporters at a COVID-19 update on Thursday.

"We will not be intimidated into cowering away from our duty ... to govern our state, not to be popular, but to do what is right."

Deputy Premier James Merlino said he had not seen "anything like this" in the almost-20 years he had been in parliament.

He said party leaders "should not at any time sidle up to far right extremists" or anti-vaxxers.

"It encourages people," he told reporters.

"Nazi imagery by people within the Liberal Party, violent extremist words, encouraging extremists - this leads to violence."

Mr Guy said he planned to tell his MPs not to attend the protests, ahead of Thursday's parliamentary session.

"You won't be seeing anyone back out there," he told reporters.

"We are concerned about some of the elements that are there. There is an extreme element now associated with the steps of parliament.

"If people are making legal threats they should be charged."

He said his family had been subjected to threats from people "associated with the extreme elements of the union movement".

"It's distressing for any politician. It's kind of unhinged behaviour. We need common sense, rationality on both sides," he said.

"No one endorses anyone associated with the hard right or hard left in any way in these protests."

Australian Associated Press