Customer surveys for TAMS 'can mislead'
ACT Auditor-General Maxine Cooper. Photo: Graham Tidy
The results of customer surveys used to measure the performance of the ACT government in some areas, including the provision of public roads, had the potential to be misleading, Auditor-General Maxine Cooper has found.
Dr Cooper raised concerns about the way results of surveys conducted for the Territory and Municipal Services Directorate (TAMS) were presented.
In response, Territory and Municipal Services changed the way it presented the information.
The surveys were undertaken to assess if the directorate had met targets for public satisfaction with a range of services, including the road network, ACTION Buses and library services.
But Dr Cooper has revealed in a report tabled in the Assembly that initial findings counted the answer "somewhat satisfied'' as if the answer "satisfied'' had been given.
The reporting method gave the incorrect impression that the directorate had met or exceeded some performance targets where in fact it had failed.
Customers had been asked to rate satisfaction on a scale of one to six, or one to 10, with ''extremely'' or ''very dissatisfied'' recording the lowest score and ''extremely'' or ''very satisfied'', the highest.
"In the statement of performance provided to the Audit Office for review, survey responses of 'four' (on a scale of 'one to six') and responses of 'six' (on a scale of 'one to ten'), which represented a 'somewhat satisfied' response, were counted as 'satisfied' in measuring the result for this indicator,'' Dr Cooper said.
"This response is lower than the 'satisfied' ratings of 'five' and 'seven' in each survey respectively.
"The inclusion of 'somewhat satisfied' responses in the reported results as 'satisfied' is potentially misleading because the response indicates that the customer is not fully 'satisfied' with the service. The customer had the option of providing a 'satisfied' response but has chosen not to do so.'
"To ensure readers were not misled by the results on customer satisfaction, the Directorate subsequently disclosed two results for these accountability indicators.''
When "somewhat satisfied'' responses were included for a question about the public road network, an 85 per cent success rate was measured, compared to a 70 per cent target. But when the "somewhats'' were excluded, the success rate fell to 68 per cent.
ACTION Buses initially submitted draft results which showed 78 per cent of passengers were "satisfied'' against a target of 85 per cent. This fell to 53 per cent when the "somewhats'' were excluded.
Dr Cooper recommended that TAMS and ACTION Buses review the methods used to calculate the results of accountability indicators.
A TAMS spokesperson said the scale it used in its surveys had previously met with audit scrutiny, and it had since changed them to comply with the Auditor-General's latest findings.
''Both the 'somewhat satisfied' and the 'somewhat dissatisfied' categories are no longer in TAMS' satisfaction surveys,'' the spokesperson said.
Dr Cooper also used the report to discuss problems ACTION Buses had encountered in accurately measuring the percentage of bus services that ran on time.
In its annual report, ACTION reported that 70 per cent of services operated on schedule but disclosed this figure did not take into account buses that broken down.
"The Audit Office therefore issued a qualified report of factual findings on ACTION's statement of performance because such services have not been 'provided on scheduled time','' Dr Cooper said.
The TAMS spokesperson said it was working with the Auditor-General's office to refine the ACTION Buses timeliness indicator to ensure it meets future scrutiny.