Brendan Nelson arrived at the memorial as a man on a mission