A report signed off by top Canberra bureaucrats has revealed stunning examples of public service inefficiency when it comes to releasing and managing data.
In one case, 11 memorandums of understanding were needed for one Commonwealth agency to share data with another, according to the Public Sector Data Management report.
In a separate example it took 18 months for two agencies to agree on a memorandum of understanding to share data, even though it only took a fortnight to do the sharing.
For others, it took several years to sign similar agreements.
Often "only a few tables" of data were produced, and linked to data sets that were then destroyed.
Prime Minister and cabinet secretary Michael Thawley, Finance Department boss Jane Halton and Communications and Arts Department secretary turned Prime Minister's chief of staff Drew Clarke had ultimate responsibility for the report.
PM&C deputy secretary Heather Smith led the team of research staff from various agencies who had the job of finding out how the bureaucracy performed on data management.
The report detailed a handful of major failings, beginning with the lack of an overarching strategy and the fact there was no clear mandate for the bureaucracy to use and release public sector data.
There were real and imaginary barriers stopping Commonwealth public servants sharing data and not enough incentives, skills and organisational arrangements to capitalise on government-held data.
"The Commonwealth does not have a strong culture of publishing data to foster economic opportunities," the report said.
"Privacy concerns and cautious interpretation of legislation are holding the APS back from making the most of its data."
"Australia's capacity to remain competitive in the digital economy is contingent upon its ability to harness the value of data.
Data on taxpayers moving from one departmental program to another was not being shared regularly enough, including instances where people had shifted from the Department of Defence to the Department of Veterans' Affairs, and the Department of Social Services to the Department of Health.
Australia lagged behind the United States and Britain in releasing public data for businesses.
Data.gov.au, the central point of access to data from Australian governments, had 6700 data sets but its British equivalent had four times that amount while the same sort of portal in the United States had almost 20 times that number.
"Most agencies do not release data via data.gov.au as a matter of course," the report said.
"Many said it simply does not enter their mind to do so."
New Zealand beat Australia when it came to using data to design policies.
The report said Britain, United States and New Zealand gave themselves a significant head start because they had invested in data policy years ago. The need to share information in those countries was preached by government ministers.
The public service will now focus on employing data scientists who have fluent coding abilities and "outside the box" thinking on how data is used, despite a shortage of data analytical skills globally.
Pockets of excellence in the public service where these skills were relatively plentiful included the Australian Taxation Office, Australian Bureau of Statistics, Bureau of Meteorology, CSIRO, Geoscience Australia, the Department of Immigration and Border Protection and the intelligence community.
The move to share more data comes after Digital Transformation Office chief executive Paul Shetler said this year that the public service was failing consumers by not making their transactions with the government easy enough.