The complexities of entering a voter's mind

There have been some understandable queries about the Patterson Research Group poll published by The Canberra Times in the days leading up to the October 20 ACT election.

Our company has a long history in opinion polling and working with The Canberra Times, having polled Eden Monaro for the 2004 and 2007 federal elections, and the 2008 ACT poll. In all three, our data was accurate to within 2 per cent or better of the final electoral outcome.

For 25 years we have conducted in the order of 300 monthly opinion polls (mostly published as WestPoll in The West Australian newspaper) as well as published pre-election polls in WA, NSW, Tasmania and the ACT. The procedures we followed in this election were the same as we have used for 15 years. Our long history has routinely produced survey results within 2 per cent of election results.

For the 2012 ACT election, The Canberra Times commissioned us to conduct an independent poll of about 1200 interviews - with a theoretical survey error over the whole ACT of 3 per cent, and +/-5 per cent within each electorate. Yet for only the second time in 25 years of Patterson Research Group's political polling, the difference between the projected and actual Labor and Green votes in particular were well outside these theoretical error limits.

The first occasion in which this apparent degree of error occurred is discussed in further detail below.

So, what happened this time in Canberra? Well, the short answer is, we don't know for sure.

We ran the survey exactly as we have run all the surveys of the past 15 years. That is, a random-dial survey overlayed with a quota management routine to ensure adequate representation of young people (males in particular). The final samples of 400 in each electorate were generated from 5069 households contacted, resulting in a response rate of 24 per cent - that is, 24 completed interviews for every 100 households contacted. (This rate compares favourably with a more typical 13-15 per cent for modern random-dial surveys.) The remainder of the 5069 calls being 33 per cent refusals, 16 per cent answering machine and 27 per cent having no reply after six call attempts.

Either something very peculiar happened in the survey, or there was a shift in voter sentiment between our polling (completed seven days before the election) and election day.

If the poll was accurate when it was taken, the electorate had to swing away from Labor and the Greens to a material degree. This is unusual, but not unheard of. Indeed, the paper's reporting of the likely outcome may have itself affected the vote.

It is on the public record (published as WestPoll) that in the 1996 election John Hewson ''could not lose'' when he tried to bring in the GST. Patterson polled the marginal seat of Canning in three waves. The first two waves showed the Liberal candidate comfortably ahead to exactly the same extent - then came the famous Hewson interview with Michael Willesee about the ironing lady and the GST components in baking a cake. It was a disaster. Our final survey showed a big swing away from the Liberals, and was within 2 per cent of the final vote.

Either we had been wrong to exactly the same extent twice, then right to within 2 per cent, using the same questions and survey format, or the Hewson interview caused a collapse in the Liberal vote. Hewson lost the election, against general consensus predictions to the contrary. (In the same election our Tasmanian polling showed a swing to Keating in the final week - again, this showed up in the election outcomes - as published in Launceston's Examiner).

We saw a similar pattern in the 2004 WA state election. We polled five marginal seats, just four days out from the election, with a sample of 1000. It provided a theoretical survey error of +/-3 per cent. This was the second occasion in which our survey result was more than 6 per cent adrift from the election result. The analysis of that event was instructive.

Our prediction of the Liberal vote was overstated by 7 per cent, Labor understated by 4.5 per cent, though the Greens and minor parties were within 1 per cent of the election result. This was well outside the theoretical sample error (+/-3 per cent) - just as was the case in last month's ACT election.

However, the same interviews included a question about the late-night trading and Sunday trading referendums being held in the same election. The results of these were within 2 per cent of the actual referendums on election day. The voting intent could not have been wrong if the predictions of the referendums were correct (our question format has not changed in more than 15 years).

So, what happened in this case? Two nights before the election, would-be premier Colin Barnett called a press conference to extol the virtues of, and costings for, his plan to create a canal to bring water from the Kimberley to Perth. There was an error in his presentation to the value of about $200 million, pointed out by reporters, and denied by Barnett on live TV. It was reported fully on all the TV news programs and in The West Australian over the final days before the election. It caused a huge swing in the last hours of the campaign, and cost him the election.

So, our survey had been correct, but there had been a wild swing in voter support. These events are unusual, but they do happen.

We are unaware of any major event in the ACT election that might have caused a similar variation. But the reports of the way the voting was trending may have itself had an impact on the election. I believe the Liberal leader has been particularly critical of The Canberra Times and our poll. The irony is that the newspaper's report that Labor was about to come back with an increased vote probably served to boost his support materially.

Evidence of poll results and reporting are difficult to come by - the cost of reliably tracking voting intent over multiple waves being extremely expensive.

But The West Australian, in previous years, has occasionally commissioned Patterson to extend our polling to track voting intent over multiple waves. We believe the results of our polling in the 1989 election, which former premier Peter Dowding won in the wake of the WA Inc scandals, did just that.

Patterson tracked voting intent over three waves for the newspaper's WestPoll series. We asked both voting intent and confidence over who would win the election.

The first of the three waves showed Labor with a 2 percentage-point lead over the opposition, and 51 per cent indicated they were confident of a Labor win. That was duly reported, and in the second survey, 63 per cent were confident of a Labor win, but their lead in the primary vote fell. When that result was published, the next polling iteration showed the confidence that Labor would win had fallen, but their primary-vote margin over the opposition had increased.

The samples were small (400 in each iteration), but suggest that there was a feedback loop in operation, enabling voters in this instance to keep a flawed government in power against an opposition widely regarded as not being up to the mark. The polling enabled voters to fine-tune their message to government without going so far as to kick it out.

In summary, in spite of the latest ACT election result on the surface being well outside the sample errors for the target electorate, after careful investigation of our processes, we have no reason to believe that our polling was erroneous. Our weighting was checked and found to be correct. The questions followed the same format (a random read-out of options) that we have used for the past 15 years.

Either the poll was wrong, or the electorate reacted to the reporting to bring the Liberals back into the picture. It is impossible to know for sure which of these two scenarios is correct. But I can report to Canberra Times readers that the poll was conducted to the highest standards, using a format that has been reliably accurate for many years.

Keith Patterson is managing director of Patterson Research Group.