The Rudd Government's ''education revolution'' is looking more and more like an extension of the Howard government's school policies. All the same elements are there choice and competition, reliance on markets, and now public reporting of school results.
The model for the new school reporting scheme comes direct from New York. Julia Gillard has been enthusing about the New York system ever since her audience with the New York schools chancellor, Joe Klein. She says she is ''inspired'' and ''impressed'' by Klein's model.
If Gillard had looked more closely, she would have seen major flaws.
The New York system produces unreliable and misleading comparisons of school performance and student progress. It is incoherent, can be used to produce league table,fails to compare like with like and is statistically flawed.
Professor of education at New York University Diane Ravitch, a former US Assistant Secretary of Education and advocate of school reporting, now says that New York's school reporting system is ''inherently unreliable'', ''dubious'' and produces ''bizarre results''.
Columbia University's Jennifer Jennings describes it as ''statistical malpractice'', ''a mess'', and based on ''highly questionable methods''. The New York Sun columnist Andrew Wolf says that it is ''an overblown grading system that already seems to be sinking from its own weight''.
New York uses an incredibly complicated scoring system, requiring two 30-page technical guides to explain. It combines a wide range of information on student achievement, student progress, student composition and school features to obtain a school grade of A to F and an overall performance score out of 100.
The process by which all this information is combined, weighted and assessed involves many highly arbitrary and subjective judgments. According to Wolf, it involves ''a bagful of subjective adjustments, bonus points and bureaucratic discretions''. It is riddled with inconsistencies.
Among the most bizarre results of the system is that high-performing schools can be assessed as failing and be closed down. For example, the elementary school PS 35 on Staten Island was graded last year as failing even though 85 per cent of its students passed the reading test and 98 per cent passed the mathematics test. It was failed because its students had shown insufficient improvement from 2006. As a result, it is a candidate for closure if it fails to improve further.
Julia Gillard says that she doesn't want ''simplistic and silly'' league tables, and will only compare schools with a similar student population. This is disingenuous. It is not possible to use the New York model to report student results in like schools without providing the scores for all schools. The New York Times and the New York Post list each school's grade and overall performance scores. It is a simple matter to rank all schools on their grade and scores in league tables.
Despite what Gillard says about the New York system, it fails to consistently compare like with like. Jennifer Jennings has pointed out that school peer groups include schools with very dissimilar demographic profiles. For example, the percentage of high-achieving Asian students in the schools of one peer group ranges from 1 per cent to 69 per cent. In another, the percentage of low-income students ranges from 12 per cent to 94 per cent.
Another major problem is that New York school progress reports do not report measurement errors for school scores and grades. As a result, its comparisons are likely to be inaccurate and misleading.
Many studies of performance reporting in England, the US and Australia have shown that a large proportion of school results are statistically indistinguishable when measurement error is taken into account. The problem is magnified for measures of student progress, or ''value added'' comparisons, where measurement error is inevitably larger.
Astoundingly, Gillard's preferred model assesses student progress on only one year's data. Yet, a study published by the US National Bureau of Economic Research shows that 50 to 80 per cent of the year-to-year fluctuations in average school test scores are random and have nothing to do with school quality. Comparisons of progress over one year are therefore highly unreliable.
These and many other criticisms mean that the New York reporting system is deeply flawed. Any system based on it will severely mislead the public and parents.
Australia and Finland are two of the highest-achieving countries in the world in school outcomes according to the PISA surveys conducted by the OECD. Neither country got there by reporting school results.
Why the Rudd Government is choosing to emulate the reporting policies of much lower-performing countries such as the United States and United Kingdom can be explained only as a triumph of ideology over evidence.
Trevor Cobbold is convener of Save Our Schools, a Canberra public education advocacy group. www.soscanberra.com
Email a friend