Bob Brown's convoy hurt Labor, says Richard Di Natale