The Coalition planned a right stitch-up when it called the royal commission into the trade union movement. But, as so often when Tony Abbott was on a mission, the project largely failed for his going too far. He had lots of ammunition. There was ample dirty linen in plain view, rumour about more, and any number of ideological axes to grind, not least about the "unfair" competition honest Liberal-voting financial brokers faced from industry superannuation funds.

But the government and, as it turned out, the royal commissioner, his team and its police task force had little experience or discrimination in selecting their targets, in assessing evidence or in judging when the well was dry. The political value of the dirt dug up has to be balanced against the cost of tasks that proved not only to be obvious duds, but ones pursued with such zeal as to confirm the subjectivity of everyone involved.

Abbott believed his inquiry could hobble the unions and hurt senior Labor figures, past, present and to come. It would be a platform for old smears against Julia Gillard, a fresh reminder to all of the putrid state of the Health Workers Union, innuendo against Bill Shorten from his trade union days, and sundry reminders that building union leaders were no gentlemen. Abbott chose for a judge an unworldly man of known views on politics and trade unions, one, moreover renowned for great learning but little judgment. Or, as it turned out, irony and introspection.

The old judge did the best he could. The words of the report are not his, except by adoption. There's a clear Heydon style of writing and talking, and this is not his voice. There was scandal enough in the management of some unions and, with that, Heydon was suitably severe, if rather over-inclined to generalise from his small and selected samples.

There were phrases chosen for headline writers and political manifestos. They were often apt. But they lost force because the report showed little capacity to grade matters by seriousness and consequence, to separate the criminal from the merely improper, misconduct from malfeasance, or to segregate self-interested betrayal of union memberships from the over-zealous prosecution of their interests. And nor could the report properly articulate the three or four big things that scream for public attention.

It's a damn pity.

Perhaps not even Bill Shorten or the CFMEU deserved Heydon, but both must bless themselves for being so lucky to appear before a tribunal whose findings could be so easily dismissed as partisan, predictable and lacking in balance.

Had the inquiry been more fair-minded and fair dinkum, working Australians would be wondering and worrying about whether the structures of the industrial and political wings of the labour movement were now fundamentally corrupted. And not just in a few rotten unions but generally. They would also be asking about the calibre, quality and fitness for office of Labor and its leaders. Even the apparently honest must be held culpable and wanting for their knowledge, complicity and apparent indifference to routine, obvious and serious abuse of power by others in the party and the trade union movement.

The headlines focus on manifestly errant individuals – people who have lined their own pockets, whether from bribery and corruption, or, more commonly, tickled the till. They have stolen from ordinary working men and women. There's scandal enough there, though the commission discovered little that was not already well known, and jumped too quickly into casting Kathy Jackson as Joan of Arc, seriously hurting its own credibility.

When the union movement, union managements, union factions, and party factions formed through union factions know of such dishonesty and rorting, and do nothing about it, they deserve what they get, politically, and in the court of public opinion. Bill Shorten is never so unconvincing as when he declares that he and the labour movement have no tolerance for corruption. Some of the worst rorters were his factional lieutenants. He's been tolerating it, overlooking it, or been culpably ignorant about it, all of his political life.

Defensive union and political folk say that plain criminality ought to have been matters for the police, not a partisan commission. But they have seriously weakened their case by years of inaction against plain rorting. And by essentially spurious distinctions between tough talk, threats and consequences in industrial negotiation and confrontation, and routine assault, blackmail and intimidation in ordinary society. The law is not suspended simply because workers, or their representatives, are defending or promoting their rights.

It was from alleged infractions in this zone that Heydon sought his headlines.

"You can look at any type of industrial union," his report said. "You can select any period of time. You can take any rank of officeholder, from secretaries down to very junior employees. You can search for any type of misbehaviour. You will find rich examples over the past 23 years in the trade union movement. These aberrations cannot be regarded as isolated. They are not the work of a few rogue unions or a few rogue officials. The misconduct exhibits great variety. It is deep-seated"."

Yes, but one could say the same of misconduct in banking and the financial services industry, in parliament and public charities and, alas, even in the police, the judiciary, religious institutions, and the public administration, often on a far grander, and personally enriching scale.

"It would be utterly naïve to think that what has been uncovered is anything other than the small tip of an enormous iceberg. It is inherently very hard to identify most types of misconduct by union officials. So far as it is typified by hard-core corruption, there is no 'victim' to complain, and the parties … have a strong incentive to keep it secret. Whistleblowers are unlikely to be found for various reasons, including a well-founded fear of reprisals. The same is true of misconduct on building sites."

Perhaps excusing the thin pickings found by his $50 million investigative team, he adds that "the very existence of a royal commission tends to cause a temporary reduction in misconduct."

"But," he says, "it is clear that in many parts of the world constituted by trade union officials, there is room for louts, thugs, bullies, thieves, perjurers, those who threaten violence, errant fiduciaries and organisers."

Not the sort one would want to put up for membership at the men-only Australian Club. But the club might be more congenial for perpetrators of some of the other conduct uncovered by the commission, if hardly given the same attention, whether by him or the headlines. Unions, like many of the other failing modern institutions, including clubs, have become entities in themselves, separated from their memberships and their reasons for existence, and often having developed interests in conflict with them.

When, for example, unions have large well-paid staffs, generous facilities, and access to enormous resources, leadership and control of such bodies is a prize. It confers power, power that can be used for good or for bad, and in furtherance of personal agendas. These may not be personal aggrandisement as such, though, as we have seen, many union politicians have seemed indifferent to venality by colleagues. It may merely be in pursuance of a political, ideological or religious agenda; it may be simple ambition and conceit that the interests of others will be helped by one's personal advancement or one's having grace and favour, rewards and punishment, with which to dispose.

Those having power are reluctant to part with it, and too many have been perfectly willing to use all of the benefits of incumbency, including the resources of members, to repel boarders and to entrench control. The rules and constitutions of many such bodies, including unions, are as much designed to avoid accountability and to reward incumbency as to make the body subject to bottom-up democratic control. Some come to judge opportunities not according to how they benefit members, but by how they benefit those in charge. Some become so taken up with the activities that are a part of a large corporate organisation, that they forget the organisation's core functions. They make judgment and do things that benefit the institution, not the memberships.

It's hardly a problem confined to trade unions. It's a problem of the club industry. Of co-ops. It's a problem of conflict between the rights and duties of corporate directors and executives compared with shareholders, and between third parties who are for some purposes agents for one side or another in a transaction. It's a problem in agencies of government, and in government itself.

It's about conflict of interest. And it's about a lack of systems for recognising and dealing with it, as well as about being transparent and open in situations where it arises. It's about capacity to hold people to account. It's about concepts of stewardship, and duty to prefer the interests of members, shareholders, clients or whoever ahead of one's own.

Trade unions, and the Labor Party, are rife with conflicts of interest. And the problems are getting worse. Bill Shorten was effectively cleared of any allegation of impropriety by Heydon, but the material involving him could serve as a good example of the problem.

It was said of Shorten that he led a fairly tame but predatory union, in great favour with employers because you could deal with Shorten. He made deals on behalf of his members that often saw their comparative conditions weakened, even as, unknown to members, Shorten and other union bosses were making side deals with employers that saw money flow into the coffers of the union, sometimes with dodgy invoices.

Sometimes such employers made direct "contributions" to campaigns designed to further the personal political careers of union executives, including Shorten himself. The allegation, in short, is that Shorten and his union executives were putting their own interests ahead of those of members. Perhaps even betraying members as they themselves prospered.

Alongside this was ample evidence of unions, including Shorten's own, being obsessed with enrolling fake or dodgy members – numbers that could be parlayed into power in industrial councils and, ultimately, into power over high decision-making in the ALP, particularly over preselections. Employers paying the dues of members, whether or not these "members", knew it. Employers paying for notional memberships – names that could be fraudulently voted in union elections. Phantoms whose notional existence justified fraudulent claims of entitlement to delegates in party councils, or to stitch up, among like-minded scallywags, the selection of allies, friends, relations and sometimes oneself into parliament or on government boards and commissions.

One didn't need Dyson Heydon to know this goes on – is in fact of the essence of the way that power is exercised in the ALP. Analogous tricks are involved in the organisation of the Liberal Party. Such skulduggery is hardly new, but it might be said to take on a new dimension in these days when political parties take tens of millions of taxpayers' money in public funding, without much in the way of serious accountability or control over the structures given the money. It is, of course, not by coincidence that the laws controlling such voluntary associations are weak and ineffectual. Nor is it any coincidence that even honest politicians are reluctant to tighten up accountability because real reform will inevitably affect power bases, and provoke retaliation.

Labor should be taking the initiative in seeking to cleanse its institutions, in both the union and the party. It should be setting and meeting new standards of being accountable to and responsible to an active membership without having to be forced. It won't, not least because Bill Shorten and a good many on the Labor frontbench would not be there if it were not for the corrupted processes, and wouldn't last a minute if the crook system was reformed.

It is for this reason that Malcolm Turnbull might be doing all working Australians a favour if he imposed some public accountability on industrial organisations and political parties, or if he helped voters (even Labor voters) see that until Labor reforms itself or is reformed by others, it is not fit for power. Even as against the Coalition.