We were told of the greatness of Malcolm Turnbull from his late adolescence, and it was freely predicted for decades that he would become prime minister. Tony Abbott was anointed, within his family, at an even earlier age. Kevin Rudd's calling, and his duty, was obvious, at least to himself, very early. Julia Gillard, John Howard and Paul Keating did not lack ambition, ability or self-belief but never gave the impression that they saw themselves fated for the top job. Mrs Hawke knew that Bob was called to govern us from the time she saw him scowling from the manger in the stable where he was born.

This sense of predestination links Bill Shorten to this pantheon, if David Marr's sharp, but by no means unfriendly profile of the Labor leader in the latest Quarterly Essay is any guide. Quarterly essays, not least by Marr, have revealed much about Australia's recent leaders. That most such essays suggest that something is missing says more about the crop than the harvester.

Shorten has worked hard for his power and his influence – and for his place on the greasy pole. He has long had his eye on the big prize. His ascension has sometimes seemed inevitable, if not necessarily deserved. He has worked his way into the right places. Cultivated the right people. Recruited people to vote for him, often blindly. Organised. Made deals, cut through to the chase. Been ruthless, dumped his friends and allies to move with former enemies. Had splendid patrons, not least at the big end of town. He's relatively new in parliamentary politics, but has been a player in party and union affairs for decades, one of the most powerful men in Victoria. If from a silvertail Jesuit school (like Abbott) steeped in labour history and shibboleths

He has helped bring down two prime ministers, both from his own party. In each case it was (at least as he sees it) from a sense of duty to the party's electoral prospects rather than mere bastardry. He took up the thankless task of rebuilding Labor from shattering defeat and, like his (Coalition) predecessor as leader of the opposition, soon looked electable for being less worse, if only ever so slightly so, than the prime minister he was facing. But since Abbott self-combusted, Shorten's victory at the next election has looked a lot less likely. But if he fails, it will not be either for want of effort or self-belief.

If Shorten is well equipped to fight Labor internal politics, he has been less certain in the public sphere. The public doesn't know what he really stands for. What he wants. What he thinks. How he might represent Australia. He has seemed scripted, unauthentic. No amount of marketing has yet persuaded us that he know him. Or that we should trust him.

No one works the Labor factional system better than Shorten, Marr says. "He is a master of the art of negotiation, a deal maker of immense skill. He betrays without flinching. The career of this man is proof that Right and Left don't mean much any more in Labor politics."

Ahead of him, if he makes it to the next federal election is a challenge he has never faced before: an absolutely open contest, facing the nation, not the party.

"Each step of the way (of his advancement in the party) has been won by deals, faction plays and the occasional walkover. He's as tough a backroom fighter as federal politics has turned up in a long time. A Shorten speciality is a brutal backroom contest that hands him a public victory 'unopposed'. He is a member for one of the safest seats in the country. He was the first leader of the party to win his position in a ballot. But that was not open either: 86 members of caucus had as much say as – and outvoted – 30,000 rank and file members of the party."

But he is at best respected in his own party, not loved, even by his closest allies. He is rich in enemies, many of whom he has stabbed in the back, but for no particularly important reason other than Shorten's pursuit of his personal interests, over the years. He does not inspire affection, let alone love or reflex loyalty. None of his colleagues is manoeuvring against him, but he has not been inspiring confidence for ages and few would rush to stand in front of any bullet headed in his direction. No one thinks he's the Messiah. No one.

The public has yet to warm to the man, to feel that they know him, or to trust his instincts. They preferred him to Abbott, but only just. He's been a leader, one way or another, for 20 years, but he has never made a memorable statement. He speaks in scripted clichés, if not in slogans. There's a lot of pure meaningless guff, and next to no passion in what he says. It's not clear what the point of him is. Or why one should prefer him to anyone else, other than because of some threat or inducement.

We do not get constant mechanical repetition of slogans, as we did from Abbott. But while Turnbull often visibly thinks aloud, or is thinking on his feet, Shorten usually sounds calculating, and scripted. He has proven almost incapable of articulating any ideal, concept or goal that one could weigh, measure, count or assay.

We found out that he was not Abbott. Perhaps, as with Abbott's campaign against Gillard, that was once enough to have him deserve power. But now he must fashion a different campaign, against a more dangerous and three-dimensional opponent. He needs a narrative, a better argument against Turnbull, and must do so within the next few months, or it will be too late. Too late for the electorate. Probably too late even for his party, even if it is not in the least clear who within it might replace him and perform better at the next election.

Last week, I commented that now as in 1996, the next Labor prime minister might not be now in Parliament. The NSW Labor Cassandra, Rod Cavalier, has pointed out that this has always been the case in Labor history over the past 100 years. Not a single next Labor prime minister was in a Parliament that survived a Labor loss of government. Scullin, Curtin, Whitlam, Hawke and Rudd got in at elections after Labor had already been at least a term in opposition. (Cavalier, only perhaps jokingly, suggests the next Labor prime minister will be Steve Waugh, sometime after he enters Parliament in 2025!)

Even as a busy union secretary, or Gillard cabinet minister, Shorten has never surrendered, or delegated, his personal factional power. In unions, he often negotiated only the outlines of the deal, leaving the details to others. This has helped him deny complicity in anything which now appears to have sold workers short, to the benefit of the union hierarchy or Shorten himself. But Shorten doesn't trust anyone, least (or perhaps particularly) fellow powerbrokers when it comes to manipulating the branch numbers, or the power of union affiliation, or the dispensation of patronage, particularly seats in Parliament.

Just what the factions he has joined, or formed, or stacked, or betrayed and reformed, have ever stood for, other than the advancement of Shorten himself, has never been clear. He has always identified himself with the pragmatic right wing of Labor politics, which has, to a large degree, defined itself merely as being against a militant left. Even when there was a (bare) whiff of ideology in Victorian factional politics, Shorten and the people he recruited, organised, rewarded, punished, used, and, usually used up and destroyed, were hardly ever fighting about ideas, or ideals. It has been primarily about personalities, and the stakes have been crude power, particularly over Labor Party preselections.

Shorten, like Stephen Conroy, another consummate player with whom he is generally, but not always in alliance, has had an enormous capacity to cultivate, recruit and deploy troops in the factional battlegrounds. But neither is followed for personality, philosophy, or charisma. Shorten is personable, and likeable enough – according to Marr his wanting to be liked is a very significant characteristic – but he is no philosopher, muse or enabler. He is strategic as much as tactical, but greatly given to Byzantine and Machiavellian play, some of which has exploded in his own face. He is not famous for his judgment about people, ideas or ideals. Many people who once thought him a best friend now detest him. It's usually his fault.

He is greatly given to making deals, forging fresh alliances, often with sworn enemies, and occasioning awful surprise among those who had thought they were his allies. He's charming but can't inspire. In private, as opposed to when on the stump, he is cynical, unblinking and without sentimentality.

"I fundamentally believe that if you empower people, you can move mountains", he tells Marr. "I fundamentally believe that if people are given a fair go, the world's a better place, people are happier individually, and society progresses. "

This is what separates him from Turnbull? Or Cory Bernardi? Or Richard Di Natale?

As Marr says, he represents no challenge to capitalism. He is AWU, and a maker of soft, not hard deals, not CMFEU. The charm of the AWU, to employers, was that they settled for a good deal less, and caused less trouble, than more militant unions. As a ministerial negotiator in the last Labor government, he was, like Rudd, a man who sometimes talked tough, but was keen to make a deal, and was regarded by some lobbies as a pushover.

He sticks to the middle ground. "His politics are blue-collar conservatism ... He has no radical designs, no great plans for reform. His goal is power for Labor and Bill Shorten, and decent administration for Australia."

Shorten says, "I know what the nation should look like in 10 or 15 years' time, and it's up to me to tell that story to the nation ... My job is to convince Australians I have a plan for the future."

Last year he promised that 2015 would be a year of ideas, but although there have been some measured policy announcements, this has been little in the way of ideas, and even less in the way of philosophies. Admittedly the Coalition was falling apart around the chaos, mismanagement and personality of Abbott and his inner office. It might have seemed bad tactics to put to voters matters which might distract from the spectacle.

Good tactics. As it turns out, bad strategy. Abbott is gone, and Turnbull is already a master of a good deal of the ground that Shorten must win. Turnbull has manifest flaws, but is many things Shorten is not. People feel they know him. He seems to be genuinely what he says he is. If he is vague, like Shorten, on just what he would do, we have a feel for the approach he would take. Shorten has yet to persuade people he is the genuine article. Or that he can be trusted to be what he seems to be. Well might Shorten complain that he has been trying to peddle a similar sort of general optimism, open-mindedness and belief that the nation's economic future lies in innovation, education and open markets. The fact is that what he has said has made little impact, nor, even it were the perfect prescription for success, has he convinced anyone that he really believes in it.

A plan, if he has one, would be nice. But a plan is not merely a group of sexy "announcables," framed through focus groups to press the right combinations of emotional buttons among voters likely to change their minds. Nor is it a combination of gestures designed to look tough, stern and statesmanlike on a few carefully chosen issues, warm and cuddly on a few others, and as close as humanly possible to the other side on everything else.

It's about a program, a choice, a discernibly different idea of where the country should be going and where it should be headed. Two years into Shorten's leadership no one, even in the party, has a reliable feel for what Shorten would do on anything, other than not much, and nothing brave. Or imaginative.

It's also a matter of character. And trust. Some sense that this leader, as compared with that, has the right instincts, the common sense and, sometimes, the convictions and integrity to do the right thing when a new situation arises. To say that Shorten has problems in this department is not to say that he's a crook, or a crock. It is that people either do not know him enough or, from what they do know, have yet seen little to show superior quality. One wouldn't, and shouldn't follow him by mere instinct. There's not, so far, enough evidence of substance. If it were there, one might expect to have seen it by now.