That's the view of counsellor David Nugent, who overcame his own problems with violence and aggression to start a behaviour change program in Victoria for men who struggled to control their temper.

He said men needed to look at themselves and their relationships to avoid contributing to the "horrible" statistics of abuse in Australia, with one in three women affected by domestic or family violence in their lifetime.

Mr Nugent said abuse commonly stemmed from men wanting to exert power and control over their partners due to insecurity or a lack of self-confidence – and emotional abuse could be just as insidious as physical violence.

"When I'm up against a wall I do one of two things, I get aggressive or I get passive, and both are violence.

"The myth in our community is that violence is physical contact but I know a lot of women who are scared for their partner to walk in the door but they've never once been hit."

He said, like a lot of men, he would have described himself as a violent man but swore he'd never hit his partner.

"We minimise the mood swings, or the grumpiness, or the roughness of being in a shitty mood.

"That's all the emotional abuse and it's putting women below the male so the male's more superior, and that's the problem in our society.''

Mr Nugent said the court system often struggled to help women who were victims of emotional violence because the offending lacked the bruises and broken bones that came with physical abuse.

The answer to stopping the violence, he said, was for men to talk to each other about their behaviour and how it could change.

Mr Nugent said while conversations about responses to family violence often revolved around boosting government funding, he advocated the importance of a "street level" approach to attacking the problem.

"I think it's important to change the culture in this country. Men need to talk to men about violence. Men and perpetrators need to step up and own it.

"They need to talk and be transparent about what they're doing and say to each other, 'Now you own it, what are you going to do about it?' "

Mr Nugent will visit Canberra this week for a screening of the ABC documentary Call Me Dad, which followed participants of the counselling program he runs in Melbourne called Heavy METAL (Men's Education Towards Anger and Life).

The documentary screening is part of the National Family Violence Summit, organised by the Tara Costigan Foundation and BaptistCare, which starts on Thursday.

Prime Minister Malcolm Turnbull and Opposition Leader Bill Shorten are set to address the inaugural conference, which aims to bring together front-line service providers, academics and political decision-makers to discuss family violence matters.

Tara Costigan Foundation chief executive Michael Costigan said family violence was "a major, major problem" across the country and it was vital the issue remained at the forefront of the political agenda.

"There hasn't, to my knowledge, been an event like this here that has brought together victims, perpetrators, support service providers, legal people, judicial people, politicians and media together to discuss these issues.''

He said a report informed by the yearly summit would be presented to the Prime Minister for an yearly statement to parliament on family violence matters.

"It's going to be an ongoing thing, from the horse's mouth, we're going to keep the government abreast of what's working and what's not working."