When the Kaikoura-Hurunui earthquake hit, at 12.02am on November 14, 2016, Kaikoura went dark. While alarms were screaming warnings elsewhere, network visibility into Kaikoura itself was completely gone.
“We couldn’t see anything into Kaikoura,” says Rob Ruiter, Chorus’ network protection manager.
The earthquake left two dead and caused widespread damage in the upper South Island on its eastern side. Wellington was rocked too.
Fixed-line services to Kaikoura were cut as the Eastern Core fibre route, run by Chorus, Spark and Vodafone, was severed in multiple places, both north and south of Kaikoura. Sections were buried under massive landslides that also cut road access to the Kaikoura township.
Cell towers were damaged, microwave links tilted in the wrong direction, and copper cables and fibre links around the region damaged.
While communications to Kaikoura needed to be restored, the telcos also had another problem. With the tripartite cable gone, Spark, Chorus and Vodafone were now relying on just one fibre-optic link for the entire South Island. Cable diversity was gone – and a weather bomb was bearing down on the region, bringing with it the threat of further landslides and potential danger for the western cable.
For Vodafone, which has an underwater cable running down the East Coast as its second South Island link, there was another issue. Called the Aqualink, this cable comes ashore near Kaikoura for a power top up, but the power was out and the repeater hut was running on battery back-up.
Getting generators to the site became Vodafone’s key priority in the hours immediately after the earthquake. By 3am, a truck carrying a generator had left Christchurch. By 4am, helicopters had been scrambled together in Wellington and Christchurch to fly more generators into Kaikoura.
“We needed to make absolutely 100 percent certain we were going to get a generator there,” says Ian Hooker, Vodafone’s head of operations.
Road-blocks saw the truck from Christchurch turned back two hours out of the city. Then, at 6am, Hooker received a call to say the Wellington helicopter couldn’t leave because of a tsunami warning.
“My response isn’t repeatable,” says Hooker.
Then the warning lifted and both helicopters arrived in Kaikoura with their precious cargo.
“We hooked up a generator and we were all good,” says Hooker.
With the Aqualink secured, Chorus, Spark and Vodafone could now collaborate and use this cable link to restore Kaikoura’s services, as well as resiliency to the wider South Island network. While the Aqualink didn’t deliver services into Kaikoura itself, modifications soon made this possible.
Horse-trading saw Spark and Chorus given capacity on Vodafone’s Aqualink cable, while Vodafone maintained its diversity by switching to capacity on the western fibre cable.
Just four days after the earthquake, fixed-line services, including broadband, were restored to Kaikoura via the Aqualink cable, with 2degrees, which already had capacity on Aqualink, also benefiting from the newly modified cable. Chorus, Spark and Vodafone all now had much-needed resiliency.
“About six months’ work happened in a week, under urgency,” says Hooker.
This was just one example of collaboration between the industry players, with engineers from competing companies all putting aside competition to work together.
Meanwhile, Kaikoura Hospital remained online throughout, thanks to a back-up wireless link that linked it through to Christchurch. When its fibre connection was cut, this back-up link, provided by 2degrees and wireless internet service provider Amuri.net, kicked in within milliseconds, providing the hospital with a full service – and the public with a much sought after wi-fi hotspot.
“The usage going through there was really significant,” says Mark Petrie, 2degrees’ former chief fixed officer. “People made full use of it. It was the only place they could get back on the internet.”
Alongside the work on the Aqualink, cell sites, copper cables and fibre were also being repaired. With roads closed and cordons in place in many areas, helicopters were used to get access to sites for repairs and for the constant refuelling the generators needed.
“We couldn't see anything into Kaikoura.”
Valuable lessons learned
The whole is truly greater than the sum of its parts
Chorus’ Rob Ruiter says the company was part of Kaikoura’s Telecommunications Emergency Forum, along with Spark and Vodafone. This meant they could show they were collaborating well, lending them weight when communicating with other agencies.
Fostering closer peacetime relationships with Civil Defence is key
Ruiter says a good knowledge of processes and who the key personnel are means people can link up immediately when there is a disaster or other major event. This ensures the value of telecommunications is understood and that telcos can get logistical and offer help more easily during restoration after a disaster.
Disasters hone business processes
Spark’s Campbell Fraser says the company learnt how to better implement services and has since shortened its delivery cycles after lighting up the new central South Island fibre cable so quickly.
The helicopter flights weren’t just used for carrying telco equipment and fuel however. The companies also pitched in with the relief effort, delivering much-needed supplies to outlying farms.
Collaboration wasn’t just happening between the telcos either. They also worked closely with Civil Defence and life-line agencies, including the New Zealand Transport Agency, to gain access to sites and ensure work on the roads had minimal impact on existing fibre.
On the ground, the telcos worked with car rental agencies to secure cars left behind by tourists as they were flown or shipped out of the area, and to provide extra transport options for workers.
One Downer’s engineer, working for Vodafone, went a step further. He found a campervan and camped near a critical site for three days, refuelling the generator and ensuring the site continued to operate.
“You had to do things like that once you got people in there,” says Hooker.
Meanwhile, work on repairing the eastern fibre route was also underway. Most of the damage was on the part of the cable
maintained by Vodafone. While engineers didn’t know it when repair work began, there were 12 breaks in the fibre, some of which was buried under massive landslides, one part deep within a tunnel that had collapsed.
Adding to the headaches were new landslips and further fibre breaks caused by the clearing of landslips – and a few inquisitive seals who became too interested in the new fibre being laid along the coast.
Fibre overlays were run through the hills behind the big slips to bypass broken sections while they were being repaired.
Spark, meanwhile, had another major project underway. The company owns a fibre link running down the western side of the Kaikoura ranges, but had never lit it up. To gain a third level of resilience, Spark began lighting it up.
Shortly before Christmas, traffic was passing over it – something Spark’s general manager of technologies, Campbell Fraser, says was “quite a feat” given work that would normally take three to four months was completed in just five weeks.