PM tried to 'silence me': ex-Liberal MP