Psychologists have slammed the Abbott government's proposed watering down of race discrimination laws, warning that any changes could damage the mental health of people from different cultural backgrounds by legitimising racism.

Attorney-General George Brandis wants to scrap the section of the Racial Discrimination Act that makes it illegal to offend or intimidate someone because of their race, colour, or national or ethnic origin.

But Australian Psychological Society president Tim Hannan said any weakening of the law could ''inadvertently legitimise'' racist behaviour.

''If the government weakens the legislative protection against racial vilification, it could provide support and justification under the guise of free speech for a broader range of racial vilification,'' he said. ''What becomes permitted by law becomes an argument for acceptable behaviour.''

Associate Professor Hannan said there was ''compelling evidence'' that racial discrimination adversely affects people's mental health and well-being.

''We have to balance the right to free speech with our obligation to protect the vulnerable from racial and ethnic prejudice,'' he said.

Studies have shown people who experience racism have poorer self-esteem, higher levels of psychological distress and conditions such as anxiety and depression.

The latest Mapping Social Cohesion survey by the Scanlon Foundation found 19 per cent of people were discriminated against because of their skin colour, ethnic origin or religious beliefs last year - up to its highest level, from 12 per cent in 2012.