Bring ministers' advisers under greater scrutiny: ex-PM&C chief