The accused Christchurch mosque gunman Brenton Tarrant has been able to send seven letters from prison, according to New Zealand Corrections Minister Kelvin Davis.
Two were to his mother and five to others, whose identity was not known, Davis told New Zealand media.
A six-page letter from Tarrant was posted this week on the website 4chan, which has become notorious as a place for white supremacists to post their views.
Davis believed the accused had received a "couple of dozen" of letters from around the world.
NZ's Corrections Department had made a mistake and the letter sent to an admirer, revealed this week, should have been withheld, Davis said.
"I've expressed my disappointment to them. They have apologised to me but more importantly for the distress it's caused to a large sector of our community.
"They need to own it, they need to make sure it doesn't happen again."
The Corrections Department has conceded the accused gunman should not have been allowed to send a letter to an admirer which was posted on a website earlier this week.
Tarrant wrote about his political and social views but says he cannot go into great detail about his regrets or feelings "as the guards will confiscate my letter if I do [to use as evidence]".
Davis was asked on New Zealand radio whether the "call to arms" in part of the inmate's letter should not have alerted Corrections that the letter should be withheld.
But Davis said the "call to arms" was a matter of interpretation.
The normal process was for Corrections to deal with mail that detailed overt actions.
Davis said New Zealand had never dealt with a prisoner like this before.
"I've asked the question, are our laws as they stand actually fit for purpose?"
Davis said he had asked for advice on changes they may need to make.
"The safety of the public of New Zealand and internationally is our number one priority."
New Zealanders would be surprised that prisoners were entitled to send and receive mail, but there were limited circumstances where it could be withheld, Davis said.
"Corrections has put a temporary stop on this prisoner's mail until we... receive absolute assurance that it upholds the safety of the public."
Davis said Corrections chief executive Christine Stevenson was very competent and should not resign. She had apologised for Corrections' failing over the letter.
"I would like to apologise for the distress that this has caused to those impacted by the tragic events of 15 March," she said.
"We have taken swift action to ensure that our processes are as effective as we need them to be. With immediate effect, this prisoner will not be able to send or receive any mail until we have absolute assurance that the process in place for screening and assessing his correspondence upholds the safety of the public, both in New Zealand and internationally."
Stevenson said the accused gunman had been in custody for five months.
"It is a fine balance to uphold our lawful obligations and mitigate all potential risks posed by the prisoner, however we are absolutely committed to ensuring that he has no opportunity to cause harm or distress, either directly or indirectly."
The accused gunman would be blocked from sending or receiving mail pending a review, Stevenson said last night.
Prime Minister Jacinda Ardern said the alleged Christchurch gunman should not have been able to send a letter expressing what she called his "hateful views" from behind bars.