Victorian students in year 9 who sat their NAPLAN test online achieved writing scores that were up to 15 points higher than their peers who wrote out their responses with pen on paper.
The anomaly has prompted the state's curriculum watchdog to contact every Victorian school to warn them against getting too excited, or disappointed, about results.
“If you are a paper school and want to compare your results to the online schools you should factor in 10 to 15 points,” Victorian Curriculum and Assessment Authority chief executive Dr David Howes said.
“And if you are a school that shifted online, just be a bit cautious in thinking you've managed to suddenly increase your writing scores.”
The gap has been put down to the greater confidence students feel when typing their responses, writing more, and being able to edit their work.
While online and pen-and-paper results in reading, spelling, grammar and punctuation, and numeracy were comparable, Dr Howes said “writing is going to have to have an asterisk on it”.
He said the discrepancy was “more than trivial but less than dramatic”.
About 20 per cent of Australian students sat NAPLAN online this year, including 8 per cent of Victorian students.
But the move online has been marred by controversy, with state governments questioning the results and the Australian Education Union calling for this year's data to be scrapped.
Preliminary data released on Tuesday shows that Australian students' writing results have slumped to their lowest levels since NAPLAN tests began.
This decline was less pronounced in Victoria, which recorded improvements in all other areas of the test.
Victorian students’ mean scores in years 3, 5 and 9 reading and years 5 and 9 numeracy were the highest recorded since NAPLAN started in 2008.
Victorian Education Minister James Merlino labelled the data “our best ever NAPLAN results” and said it showed that the government's Education State agenda was having an impact.
“This is about both excellence and equity,” he said. “This is the evidence that what we are doing is working.”
Mr Merlino said the proportion of Victorian students in the bottom three bands of performance had decreased, while those in the top two bands increased in primary school reading and year 9 numeracy.
He stressed his concerns about expanding NAPLAN online to more schools.
“We were supposed to get the data quicker and that didn't happen. All of the results were meant to be comparable and that didn't happen when it came to year 9 writing.”
Australian Curriculum, Assessment and Reporting Authority chief executive Robert Randall said that on a national level, there had been significant improvements across a range of areas since 2008. But he said the 2018 results were stable when compared with last year's results.
He said the transition to NAPLAN online had been smooth, and he hoped all students would sit the online test by 2020.
“It provides us with more accurate information,” he said, referring to the test's adaptive features.
Unlike the pen-and-paper version, the adaptive online test becomes easier or more difficult depending on how many questions students answer correctly.
Clayton North Primary School was among the 124 Victorian schools that signed up for NAPLAN online this year.
About 90 per cent of students are from a language background other than English, and many have parents who are completing PhDs at the nearby Monash University.
Assistant principal Edward Strain said students found it a lot less stressful than the pen-and-paper version.
''Placing a paper-based test in front of a student leads to that more intense stress. Placing a computer or an iPad in front of them is something they are doing day to day anyway.”