Australian soldiers who have been moonlighting as security guards at offshore immigration detentions camps, including Manus Island in Papua New Guinea, are quitting their jobs and heading home.

But military authorities in Canberra deny they have ordered the men back to Australia.

Fairfax revealed in April that serving soldiers were using their leave entitlements to work for security companies at detention camps in Manus and Nauru.

There were also allegations of widespread use of illegal drugs among the security workforce at Manus, and that several guards had been sacked for "innappropriate behaviour".

In the wake of the Fairfax story, Manus insiders have reported that the serving military, many of whom had not told their superior officers they were working in the volatile detention system, were quitting their jobs with contractor Wilson Security and heading back to Australia.

One source said the departures were leaving the remaining security force stretched.

"They are now very short-staffed as the army has pulled all serving [staff] from here due to the media getting a hold, and heaps have also pulled the pin," the Manus insider said.

The Defence Force confirmed it had given the green light for seven serving soldiers and an unspecified number of air force personnel to work as guards at Australia's network of offshore detention centres.

But the Defence Department "media opps" unit in Canberra said neither the army nor the air force had ordered the off-duty troops home.

"The Australian Army is aware of members undertaking approved secondary employment in off-duty hours with Wilson Security on Manus Island," the media unit said in a statement.

"Army, however, has not issued a directive requiring these members to cease their approved leave and return to Australia.

"There have been no changes made to air force policy concerning off-duty employment."

A spokesman for services giant Transfield, which runs the camps on behalf of the federal government, also denied soldiers had been ordered home. 

"On behalf of Wilson I can confirm there has been no communication to this effect," a company spokesman said.

The Immigration Department did not answer questions about whether it had intervened in the wake of April's story but issued a statement on behalf of its minister Scott Morrison.

“The government expects that service providers conduct themselves lawfully and consistent with the service standards set out in their contract," the statement said.

The allegations of steroid use and bad behaviour by guards were part of a series of bad news stories for the troubled centre, the scene of a night of violence in February that left one asylum seeker dead and 62 injured.

Several Wilson guards have been sacked for inappropriate conduct since the company took over the Manus security contract in the wake of the violence, including one man dismissed after a bizarre incident of sexual harassment of a disabled asylum seeker.

The security company has defended its record, saying its workers were thoroughly drug-tested before they were hired but were not asked to disclose their military status.

Wilson Security was hired in February for the Manus contract, soon after detention centre operator Transfield was chosen to replace G4S, which recruited its own security guards.

Transfield is being paid $61 million a month to run the centres on Manus Island, which has about 1300 asylum seekers, and Nauru, where about 1200 people are held.