It was an unlucky day for all involved in fixing numerous faults that caused widespread outages on Vodafone and Optus mobile networks on Thursday.
Not only did the main fibre link connecting Western Australia to the rest of Australia fail, but the redundancy link also carked it.
The fault then led to Vodafone's network being overloaded, leaving customers unable to make or receive calls Australia-wide.
The issue? A combination of a faulty repeater on a primary fibre link and a back-up cable failing.
A completely different "system error" prevented some customers on Optus's network from making and receiving calls and using data. That issue? IMEI numbers - unique numbers assigned to mobile phones - were accidentally blocked.
"It's pretty rare to have simultaneous outages like this," said a spokesman for Nextgen Networks, which provides one of the links Vodafone uses connecting WA to the rest of the country.
"So what happened was that another carrier had a problem on their network which meant that our back-up link [run by them] was down on Thursday," the spokesman said.
"Then ... on our primary link we had a repeater failure."
The repeater, at Bullabulling, a remote town 455 kilometres east of Perth as the crow flies, failed at about 10.55am AEST. Between then and 12.25pm, a change was made remotely to the cable the repeater was on to make it usable again. Then, at 2.40pm, the repeater was fixed by a technician who drove from Kalgoorlie to Bullabulling (a 40-minute drive).
"We definitely sympathise with our customers and the end users affected," Nextgen's spokesman said, who emphasised that the fibre link was one of the longest in the world.
"It's subject to harsh conditions and from time to time equipment fails," he said.
"There's no power there - everything is solar powered - and it's relatively risky in general."
Vodafone said the national issue occured after the link went down.
"The reintegration of the West Australian network to the national network triggered an unexpected overload in our voice switching network," a Vodafone spokeswoman said.
"Our engineers had to restart a number of nodes to stabilise them, after which services to our customers were restored."
As an apology for the failure, Vodafone has offered customers free mobile data this weekend.
"We know our customers had a less than perfect network experience and we want them to know we're sorry," Vodafone's chief technology officer Benoit Hanssen said in a statement.
Optus said an isolated group of customers were affected by its issue and that it was contacting those customers directly to "say sorry and give a credit".
It's unclear whether companies that resell Optus's network - including, but not limited to, TPG Mobile, Virgin Mobile, Amaysim, iiNet Mobile and Node Mobile - will compensate customers.
Vodafone customers will be able to use "as much data as they like" within Australia from midnight AEST Friday until 3am AEST Monday. Surprisingly, the offer comes with no caveats, which will likely result in a huge surge in data activity - the ultimate test for Vodafone's rebuilt mobile network since the network problems the telco faced in 2010.
The offer means Vodafone customers can theoretically download 1800 gigabytes of mobile data over the free period if their speed remains at 100 megabits per second and they're using a 4G handset, according to Cybershack. Considering the telco charges 10 cents a megabyte for excess usage, the maximum amount of data is $184,320 worth of value.
Despite the free data offer, some Vodafone customers are still not happy.
Mother-of-two Monique Atkinson, of Round Corner in NSW, said her 5-year-old daughter, Hailey, was sick at school and she was unaware of it as her school could not get in touch with her.
"I'm kind of hoping for a proper apology and some money to be taken off the bills that have been astronomical of late," Mrs Atkinson said.
Mrs Atkinson said she and her husband would be leaving Vodafone when their contracts expired.
"They've had terrible coverage for a while now. They say they have great coverage in most of Australia and that they're not a bad company. Yes they do great deals for the price, but the coverage is still not that great. My husband drives trucks and I can barely reach him."
Optus reseller Amaysim said many of its customers weren't affected by the issue. "We'll work with any customers impacted on an individual basis regarding goodwill," a spokesman said.
Virgin Mobile said it would also work with customers on a case by case basis.
iiNet Group - which runs iiNet Mobile and Node Mobile - has yet to respond to a request for comment.