Scott Morrison has called signs outside departmental toilets 'ridiculous'