Senior leaders in the Prime Minister and Cabinet department's Indigenous Affairs Group have based themselves in Canberra's dress circle, nearly 10 kilometres away from their rank-and-file workers, who are still reeling after repeated restructures to their workplaces.

The internal problems have emerged on the eve of the Prime Minister's trip to Arnhem Land, part of his pledge to be "a prime minister for indigenous affairs" that also included the takeover by PM&C of indigenous functions from several other government agencies.

The department admits that things are "difficult" but says it is working "towards building a cohesive department" after forced mergers saw it try to absorb an extra 1800 public servants.

The Prime Minister's office did not respond to requests for comment.

Do you know more? Send your confidential tips to ps@canberratimes.com.au

The hundreds of public servants still working in the PM&C's Indigenous Affairs Group are trying to overhaul programs and transfer key responsibilities to regional offices in line with Mr Abbott's promised shake-up of services.

But departmental sources say the work is going on against a background of upheaval and disruption after several restructures and 236 redundancies, including senior executives, and middle-managers distracted by being forced to apply for their own jobs.

"Staff are still unclear how the new program arrangements will be assessed and managed," one PM&C staffer said.

"The disorganised state has meant that indigenous groups and other stakeholders have not been consulted.

"Most 2014-2015 money is not available as many existing contracts were automatically extended for 12 months.

"There are some funds for indigenous education projects but this will be well short of demand.

"Staff morale is in the doldrums in IA group."

There is also unhappiness among Indigenous Affairs staffers that their senior leaders, Associate Secretary Liza Carroll and Deputy Secretary Richard Eccles, have based themselves at PM&C's Barton HQ in Canberra's leafy inner-south, while most of their subordinates continue to work 9.5 kilometres away in unfashionable Woden.

But a PM&C spokesman defended his department's performance, saying it was doing its best after being quadrupled in size and having to absorb 1800 extra public servants from nine other agencies, with the Prime Minister also taking over as Minister for Women.

"Most of the staff were in the Indigenous Affairs area, but the deregulation function and the Office for Women are also new to PM&C," the spokesman said.

"Change of this magnitude is always difficult, and we are continuing to work towards building a cohesive department.

"The effect of the efficiency dividend and the streamlined programme structure has seen a reduction of staff.

"236 staff have taken a voluntary redundancy."

The spokesman said all current funding contracts were being honoured and service providers whose contracts ended on 30 June 2014 and "who had an expectation of ongoing funding" have had their contracts extended.

"The government will work in partnership with Indigenous communities to tailor action in the priority areas of schooling, jobs and community safety," the spokesman said.

"Under these arrangements, senior decision-makers will be closer to communities.

"Associate Secretary Liza Carroll and Deputy Secretary Richard Eccles work in both Barton and Woden, as appropriate."