I remember a remark delivered with a wry grin by a senior figure close to the last Labor government: "What if they're just not very good?"
Could it be that despite all the data crunching, the focus groups, the media toadying and defensive drops to News Corp, they just weren't strategically up to it?
Battles were won of course, like plain-paper cigarette packaging, the GFC response, the NBN, the NDIS, and the carbon price.
But the twin forces arrayed against all governments - AKA the media and the Opposition - ensure that it's the negatives which leave the deeper impression.
For reforming parties, those difficulties are more acute because change raises fears, creates potential losers and hands opponents the stick with which to belt you.
The bar is set lower for ambition-free zones like the government Scott Morrison was the third to inherit.
It should have been his policies but it was mainly Morrison's "not-my-job" slipperiness that did him in.
Even so, a divided party that had necked two prime ministers coaxed three terms out of voters. Go figure.
Against this under-performance, Labor's current slump feels wildly over-wrought. Yet commentators, and some MPs are discussing what seemed ridiculous just weeks ago.
Here's the doyen of the Canberra Press Gallery, Michelle Grattan, on Friday: "... while a few months ago Dutton was considered simply "unelectable", now that view is more hedged. If the government's position doesn't improve substantially, people will take a more serious look at the hard man from Queensland, and speculate about what sort of prime minister he'd make."
For David Crowe in the Nine papers, Labor might have landed its election commitments but it is "agonising over what to do next".
"The defeat of the Indigenous Voice referendum marked a key moment when the voting public dismissed what Labor was selling.
"Labor has looked bruised ever since, slow to act on challenges such as the High Court ruling on detainees."
In his midweek column, Shaun Carney argued Labor's public support had dived due to "a lack of policy ambition" and the absence of "a coherent and comprehensive explanation for the government's reason for being".
Rapid ennui is the new normal in a post-truth/social media age in which procedural government is incompatible with a shallow and impatient politics.
Where voters used to pick and stick, the vogue now is select and reject in a febrile discourse beset with cynicism and buyer's remorse.
Against the comforting truth that the last government to get just one term was elected in 1929, is the sobering fact that the last prime minster to win a second term was John Howard in 1998.
Still, records are made to be broken, as the Aston byelection showed. Even those we hold up as successful lost seats at their first check-back-in with voters - Hawke in 1984 and Howard in 1998. As did Labor in 2010 (Rudd-Gillard), and the Coalition in 2016 (Abbott-Turnbull).
In short, voters often cavil with the new broom they ordered which proves less decisive than imagined.
The best insider explanation I've ever heard of this was delivered by Jessica Rudd at the National Press Club on Wednesday.
"First terms are notoriously haunted by the spark and bright ideas of a recent, and all-too-eager opposition desperate to get there, but discovering, post-election, that things are 'always' easier said than done," she said.
"The rusty cogs of bureaucracy creak and 'events' at home and abroad 'always' get in the way of delivering services.
"Honeymoons are declared over. Polls narrow. And the three years' worth of parliamentary sand slips through the hour-glass quicker than you can say Yarralumla."
Of course, this is numerically more problematic for Albanese simply because he can afford to lose just two seats before he's heading into minority territory.
While J-Rudd's reduction is brilliant in generic terms, a more particular perspective was given by one MP who sees the problem as a natural corollary of Labor's calculated pre-election modesty in 2022.
"The challenge when you build your re-election argument exclusively around "promises delivered" is two-fold.
"First, if you ran a safe, small target opposition, you will end up running a safe, small target government.
"Second, it risks over-weighting things already ticked off rather than explaining what we're going to do from here to help ordinary voters."
In other words, sticking only to your promises, the MP said, even when major circumstances change - think a dozen-plus interest rate hikes - can feel virtuous but look inactive.
It's an interesting point and, I suspect, one the government should take to heart.
Yet it seems to me more is being made of Labor's short-term difficulties than will prove to be important over time.
Dutton, who is the most negative opposition leader since Abbott and even less philosophical, has done little more than surf the government's break.
Not only does he eschew the accountability of Canberra press conferences, he is yet to make a serious long-form speech or lecture setting out his ethos, his alternative Australia.
That said, the government will go into the summer break needing to reassess and as Dutton himself has predicted to his party room, will likely recalibrate for its run to an election either late 2024 or early 2025.
Albanese holds that initiative and he has a level of unity that eluded Kevin Rudd and Julia Gillard. The question is, can he capitalise on it?
- Mark Kenny is The Canberra Times' political analyst and a professor at the ANU's Australian Studies Institute. He hosts the Democracy Sausage podcast. He writes a column every Sunday.