However Australian Security intelligence Organisation director-general Mike Burgess said it was unclear what is driving that growth.

"We do not fully understand why we're seeing a growth. This is a matter that ASIO has been looking at for a good number of years," Mr Burgess told Senate estimates on Monday.

"In the last three years, we've seen a rise in activity and strengthening and there's a few trends there but we do not fully understand why this is occurring."

Mr Burgess last week used the agency's first ever "threat assessment speech" to warn of small cells regularly meeting in suburbs around Australia to "salute Nazi flags, inspect weapons, train in combat and share their hateful ideology".

"We expect such groups will remain an enduring threat, making use of online propaganda to spread their messages of hate," he said at the time.

However Mr Burgess dismissed the need for a centralised office to investigate right wing extremism.

Germany recently announced it would set up a central office to tackle far right extremist activities.

But Mr Burgess said Germany also had a "history in this space" - a reference to Nazism - which had informed its response to right-wing extremism.

"This one is growing concern for us, but it ... is not yet currently my principal concern," Mr Burgess said.

Australia also has no right wing groups listed as terrorist organisations.

Mr Burgess said ASIO constantly reviewed the prescription of certain groups.

"Whilst it's under constant review, I can't go into full details of those particular matters," he said.

Islamic extremism remained ASIO's primary concern, Mr Burgess said.

Meanwhile senators have complained about the use of the term "right wing" to describe extremism.

Liberal senator Concetta Fierravanti-Wells said "there are many people of conservative background who take exception to being charred (sic) with that brush".

"Right is associated with conservatism in this country. And I think that you do understand that your comments, particularly when you refer to them solely as right wing, has the potential to offend a lot of Australians," Senator Fierravanti-Wells said.

READ MORE:

Mr Burgess said he did not intend to offend anyone but Neo Nazi and similar groups had "long been given the label of right wing, not by ASIO".

"It's unfortunate that we refer to it as right wing extremism, but in the absence of something else, which maybe we should look at a different label," Mr Burgess said.

One Nation senator Malcolm Roberts suggested left and right wing be replaced with "control versus freedom."

"It seems to me that while ever the terminology is confused, your job will be more difficult in communicating to the Australian people," Senator Roberts said.

Mr Burgess said he was "not sure I quite go for freedom versus control."

"This is about individuals or groups that promote communal violence or politically motivated violence," Mr Burgess said.

"I agree it's unhelpful to assign them to a part of the political spectrum because actually you know being on the political spectrum is a great part of our democracy."

Home Affairs Minister Peter Dutton last week labelled Islamic terrorists as left wing extremists.

However he told Insiders on Sunday the point he was making was he was "completely blind to somebody's religious beliefs".

"I don't believe we should tolerate people acting outside of the law wherever they are on the spectrum," Mr Dutton said.