Labor's primary vote plunged seven per cent in the coal-mining seat, with the National Party comfortably returned to power.

The result has reopened a festering wound within federal ranks over whether the Labor Party is staunch enough in support of workers in fossil fuels.

Mr Albanese is keen to focus on areas like Muswellbrook, which is home to most of the electorate's coal miners, where Labor received a positive swing.

He also cautioned against drawing too many national conclusions from a state result, pointing to Labor's thumping wins in Queensland and Western Australia.

"Let's get a bit of perspective here, quite frankly," Mr Albanese told reporters on Monday.

"This is a seat that Labor has not held in the last nine decades at any time, at any time whatsoever.

"This is a seat whereby, frankly, a couple of elections ago we would have struggled to find people to hand out how-to-vote cards."

Mr Albanese said the National Party's primary vote in Upper Hunter had plummeted by 24 per cent in recent years, while Labor's primary was not that much to begin with.

Outspoken backbencher Joel Fitzgibbon, the federal member for the Hunter, said the result was devastating for the Labor Party.

Just one in five voters put Labor first on their ballot papers.

Mr Fitzgibbon warned Labor was on track to lose the next federal election without a significant shift and threatened to quit if the party did not 'wake up to itself'.

"Federally, if Labor can't persuade not just mine workers but everyone in those regions whose jobs are dependent on mining, that we stand with them, you can expect a similar result whenever Scott Morrison goes to the polls," he said.

Meryl Swanson, who holds the neighbouring electorate of Paterson, also believes the by-election result was linked to Labor losing touch with blue collar voters.

But Pat Conroy, whose seat of Shortland is also in the coal region, rejected suggestions the result had federal implications.

Mr Conroy manned a booth on Saturday and said not one person raised any federal issues with him.

The coalition is expected to target all three Labor-held seats at the next federal election.

Upper Hunter voters who deserted Labor did not go to the coalition, but rather independents and One Nation.

The Nationals only polled 31.4 per cent of the primary vote.

One in three voters were happy to ignore both major parties, siding with independents and minor parties instead.

This could have major implications for the Senate if the trend was replicated at the next federal election and could also deliver a much more diverse crossbench in the lower house.

Finance Minister Simon Birmingham has cautioned against drawing too many national implications from the Upper Hunter result.

"It was a state by-election and I would treat it as such," he said.

"It had a range of different factors at play and certainly many of them were very much local, very much state politics."

Senator Birmingham said it was far too soon to speculate on how many Hunter Valley seats the coalition could win at the next federal poll.

"It's a long way away to start making those sorts of predictions but we will put up a strong fight in those regions," he said.

"They are regions which clearly have large numbers of working Australian families who want to know they've got a government that is with them and backing them."

Australian Associated Press