Keith Lowry says too few organisations can even identify their most crucial data – their "crown jewels" – let alone control and monitor tightly which employees access and transfer it.

He also says reliance on the security vetting of recruits is far from adequate.

"Here's the interesting thing: all of the spies that have been caught in [US government agencies] passed background checks," he says.

"They all decided, at some point in time, that they were going to do something that they had agreed not to do or swore not to do, and they broke their promise.

"A background check will not predict future behaviour."

Mr Lowry, who will be in Canberra this week to brief intelligence and security officers, led the team that assessed the damage caused by former CIA analyst Edward Snowden's leaks of National Security Agency secrets.

Before that, Mr Lowry helped investigate Chelsea (then Bradley) Manning, who was convicted of leaking about 750,000 military and diplomatic documents to anti-secrecy group WikiLeaks.

Mr Lowry says one difficulty in vetting staff effectively is the range of motivations that can undermine an employee's loyalty.

"Motivations are across the board. Most of it has to do with money," he says.

"But then there are people who are disgruntled employees; there are people who just are malicious in nature; there are those who are dissatisfie with their management – they get a new job and they dislike the old one, so they leave behind a mess."

The Australian National Audit Office released a report last that criticised the Commonwealth's security clearance regime.

It said the Australian Government Security Vetting Agency had been hampered initially by a lack of funds. However, despite significant recent investment, it remained inefficient, its computer systems were inadequate and it could not vet staff quickly enough.

The audit report also cited high demand within government for vetting: some workplaces required all staff to hold security clearances because they allowed every employee to access classified information.

Mr Lowry, who now works for Australian data investigation business Nuix, said modern technology made it near impossible for companies or government agencies to control all data they created or held.

But they could protect themselves by determining which data was most sensitive and developing practices that prevented that information from being breached by malicious staff, loss or theft of devices, or IT system errors.

"Insiders can cause greater damage compared to external cyber attacks but many organisations still have poor strategies in place to prevent a serious and costly insider data breach from occurring," he said.

"Executive teams need a new approach to these risks because old security models and IT systems, which focus on outside cyber attacks, provide very little defence against an inside job."

In a speech last year, Attorney-General George Brandis also noted the threat posed by wayward public servants, saying the "trusted insider can access – on an unprecedented scale today – massive amounts of sensitive information through our networked computers and copy and transfer it with ease".

"That is why the two largest breaches of Western intelligence information have occurred only recently," he said.