The result was scorched-earth file corruption that spread automatically, rapidly, and indiscriminately. “To date, it was simply the fastest-propagating piece of malware we’ve ever seen,” Craig Williams, a researcher at Cisco’s security division Talos, told me. “By the second you saw it, your data center was already gone.”
NotPetya was, in fact, more pestilential than likely even its creators intended. Within hours, it would spread beyond Ukraine and out to countless machines around the world. It crippled multinational companies including Maersk; the pharmaceutical giant Merck; FedEx’s European subsidiary, TNT Express; the French construction company Saint-Gobain; Cadbury’s and Nabisco’s food-industry parent company, Mondelēz; and the U.K. manufacturer Reckitt Benckiser, whose products include Durex condoms and Lysol disinfectant. In each of those cases, it would cause hundreds of millions of dollars in damages. It even spread to Russia—the cybersecurity community’s immediate prime suspect for NotPetya’s origin—striking victims like the state oil company Rosneft, the steelmaker Evraz, the medical technology firm Invitro, and Sberbank.
But at a national scale, no country would feel NotPetya’s effects quite like Ukraine. Even as the worm’s tentacles were spreading out from its initial victims into networks across the globe, the mass of infections at its core was busy eating Ukraine’s digital infrastructure alive.
25
NATIONAL DISASTER
That same Tuesday morning, Serhiy Honcharov began his long, strange commute, just as he did every day. In the far-north Ukrainian town of Slavutych, the bespectacled, taciturn engineer boarded a train that headed east, crossing the Dnieper River and then dipping briefly over the Belarusian border. The train passed through forty miles of landscape that had seen no human influence in more than three decades. Trees and grass grew as tall and wild as they had in a prehistoric era. From his train window, he saw deer and flocks of birds, all of which seemed to prefer the radioactive remnants of humans’ civilization to humanity’s presence. Finally the train crossed the Pripyat River and arrived at a station that served only one destination: the cleanup site for the Chernobyl nuclear reactor.
At the station, Honcharov put on his blue uniform and boarded a bus that took him to the building where he worked in the sprawling Chernobyl complex, with a footprint larger than a dozen football fields. On one end of the grounds stood a gleaming structure that staff called the Arch, a massive hangar-like building taller than the Statue of Liberty and wider than the Roman Colosseum, designed to contain the supremely toxic ruins of the Chernobyl nuclear reactors. Inside, there remained nearly two hundred tons of uranium fuel, barely touched since the initial, tragic 1986 cleanup effort. The monumental mission of the Chernobyl facility’s staff was to pull that fuel out of the Chernobyl reactors with cranes and safely bury it on a site nearby in the Exclusion Zone, a process of such scale and delicacy that it’s projected to take until 2064.
Honcharov arrived at a building on the opposite end of the complex, where he served as the Chernobyl facility’s IT director. He had been in his office for only two hours when he started to receive calls that something was going terribly wrong: Across the site’s dozen-plus buildings, staff were seeing their screens go dark, then show NotPetya’s ransom messages after they rebooted. Honcharov hurried into the room full of systems administrators next to his office, who were shocked by the speed of the malware tearing across their network.
Within seven minutes, they’d made the decision to turn off all the thousand-plus Windows machines at the entire Chernobyl site. The critical functions of the equipment dealing with radioactive waste were disconnected from the infected network and wouldn’t be affected. But all the computers for the site’s administration and communication with the outside world were about to go dark.
A man’s voice read a message over the emergency loudspeaker system that reached every building in the complex. Thirty-one years after Chernobyl’s world-shaking nuclear disaster, the site reverberated with a warning for a very different sort of meltdown. “To all staff members, immediately turn off computers and unplug network cables. Await further instructions.”
* * *
■
Around the same time, back in the capital, an IT administrator for Ukraine’s Ministry of Health named Pavlo Bondarenko was watching NotPetya’s wave begin to crest across social media, seeing the same ransom screens appear again and again on Facebook and Telegram. Bondarenko, a twenty-two-year-old, six-foot-seven tech consultant with a mane of curly blond hair and a build that resembled a pro wrestler more than a government employee, watched the growing signs from his desk in the ministry’s office. He sensed that Kiev’s government agencies would be next.
Bondarenko called the health minister, Ulana Suprun, and made an unthinkable proposal: Unplug the ministry’s entire network, responsible for everything from payroll for workers to cataloging stores of medicines to the national database of organ donors and recipients. “Save the data,” as he described the approach he pitched to Suprun. “Don’t think about the consequences.”
Suprun agreed. Bondarenko and his colleagues started frantically disconnecting the ministry’s computers and turning off its network links. Within hours, practically every other federal agency in Ukraine had either followed suit or else watched NotPetya tear through its systems, paralyzing everything in its path. “The government was dead,” summarizes the Ukrainian minister of infrastructure Volodymyr Omelyan. Soon, NotPetya had hit Ukraine’s national railways, taking down its ticketing system just as in the late 2016 attacks. It tore through Kiev’s Boryspil airport, blacking out the scheduling screens across its terminals.
By 1:00 p.m., NotPetya had begun to topple another major pillar of Ukrainian society: the post office. The first ransom screens began to appear in the service’s iconic white stone headquarters on Kiev’s Maidan. Within an hour, Oleksandr Ryabets, the national postal service’s director of IT, was pacing the halls, speaking on a conference call with the service’s CEO, Igor Smelyansky, who had been at a meeting in Lviv, in the west of the country. They’d spoken for just a few minutes when Smelyansky gave Ryabets the order to shut down the agency’s entire national network.
In Ukrainian society, the postal service’s IT systems are responsible for more than mere mail. They also handle money transfers, newspaper subscriptions, and, perhaps most critically, pension payments that support 4.5 million retirees, along with the payroll of the postal service’s own 74,000 employees and the dispatch system for 2,500 postal trucks.
Ryabets, a balding, central-casting career civil servant with a permanently weary expression, paused for a moment to process his boss’s unthinkable directive to turn off those vital digital services, which in many cases would mean handling their gargantuan complexity with pen and paper. Then he and his staff spent the next hour on their phones, spreading the shutdown order out to twenty-five regional headquarters responsible for 11,500 branch offices and a total of 23,000 PCs and servers. (In fact, they’d later find, the move to unplug had come too late: More than 70 percent of the postal service’s computers had already been infected, a mind-boggling disarray from which it would take months to recover.)
That afternoon, when the last of those offices had received the message and the shutdown was complete, Ryabets remembers feeling an eerie, death-like quiet descending over the building. “There was a kind of shocking emptiness,” he said. “It was like you’re dancing at a disco party when suddenly the music turns off, and everything is silent.”
* * *
■
Around 6:00 p.m., ISSP’s chairman Oleh Derevianko finally left the roadside restaurant where he’d unexpectedly spent the day fielding calls from shell-shocked clients. Before getting back on the road, he stopped to refuel his car. That’s when he discovered that the gas station’s credit card payment system had been taken out by NotPetya too. With no cash in his pockets, he carefully eyed his gas gauge, wondering if he
had enough fuel to reach his village.
Across the country, Ukrainians were asking themselves similar questions: whether they had enough money for groceries and gas to last through the blitz, whether they would receive their paychecks and pensions, whether their prescriptions would be filled.
One of them was Pavlo Bondarenko, the twenty-two-year-old Health Ministry and IT administrator. Bondarenko left his office around 7:00 and headed out into the still-light summer evening. But when he swiped his contactless credit card at the turnstiles of the Arsenalna subway station nearby, he found that it was unresponsive. Yet another NotPetya casualty.
He had no cash to buy a token. So he headed out into the neighborhood to use a nearby ATM, only to find that it was dead. So was the next one he tried. And the one after that. On his fourth try, he found one working cash machine, with a long line and a tiny withdrawal limit.
Bondarenko took out enough cash to buy a metro ride home, then emerged in the Obolon neighborhood, where he lived in the north of the city. On his way to the apartment he shared with his mother, he stopped in a grocery store to buy enough milk, meat, and bread to last a couple of days. At the checkout line, he found that there, too, the point-of-sale systems were down, and cashiers were taking only cash. He didn’t have enough bills left. So he went back out into the street and repeated his desperate hunt for cash, trying another five ATMs before he was able to find one that worked.
Later, after he’d finally gotten home with his groceries, Bondarenko sat down in front of his computer to pay his Kyivenergo electricity bill. He found, in one final, comic frustration, that the site was broken; the electric utility’s payment system had been pulled off-line.
“It felt like a bad end-of-the-world movie. You’re disoriented. You can’t understand what to do next. You feel like you’ve lost an arm and can’t function properly,” Bondarenko said. “Life went very fast from ‘What’s new on Facebook?’ to ‘Do I have enough money to buy food for tomorrow?’ ”
* * *
■
Even then, NotPetya’s rampage through Ukraine wasn’t over. At 10:00 p.m., Mikhail Radutskiy, the president of a group of Kiev hospitals known as Boris Clinic, was brushing his teeth in the bathroom of his house in the western suburbs of Kiev when he got his NotPetya call. He drove into the city to find that his hospitals had been hit hard: Virtually all their Windows machines were now encrypted, though medical equipment running Linux and IBM operating systems had been spared.
All upcoming appointments had to be canceled. The GPS system for locating the hospitals’ ambulances was dead. The IT administrators had a full backup of their systems from three days earlier. But every test that had been performed since then, from blood analyses to MRIs to CAT scans, would have to be redone.
Radutskiy didn’t go home that night. By morning, angry patients with canceled appointments were collecting in the clinics’ lobbies, hallways, even the waiting room outside his office. “It was a mess,” Radutskiy told me simply. “It was chaos.”
In sum, by the end of June 27, NotPetya had struck at least four hospitals in Kiev alone, along with six power companies, two airports, more than twenty-two Ukrainian banks, ATMs, and card payment systems, and practically the entire federal government. According to ISSP, at least three hundred companies were hit, and one senior Ukrainian government official would later estimate that a total of 10 percent of all computers in the country were wiped; the country’s internet was literally decimated. “It was a massive bombing of all our systems,” Minister of Infrastructure Omelyan said.
That night, the outside world was still debating whether NotPetya was criminal ransomware or a weapon of state-sponsored cyberwar. But ISSP’s Oleksii Yasinsky and Oleh Derevianko had already started referring to it as a new kind of phenomenon: a “massive, coordinated cyber invasion.”
Meanwhile, amid that digital epidemic, one single infection would become particularly fateful for the shipping giant Maersk. In an office in Odessa, a port city on Ukraine’s Black Sea coast, a finance executive for Maersk’s Ukraine operation had asked IT administrators to install the accounting software M.E.Doc on a single computer. That gave NotPetya the only foothold it needed.
26
BREAKDOWN
The shipping terminal in Elizabeth, New Jersey—one of the seventy-six that make up the port-operations division of Maersk known as APM Terminals—sprawls out into Newark Bay on a man-made peninsula covering a full square mile. Tens of thousands of stacked, perfectly modular shipping containers cover its vast asphalt landscape, and two-hundred-foot-high blue cranes loom over the bay. From the top floors of lower Manhattan’s skyscrapers, five miles away, they look like brachiosaurs gathered at a Jurassic-era watering hole.
On a good day, about three thousand trucks arrive at the terminal, each assigned to pick up or drop off tens of thousands of pounds of everything from diapers to avocados to tractor parts. The trucks start that process, much like airline passengers, by checking in at the terminal’s gate, where scanners automatically read their container’s bar codes and a Maersk gate clerk talks to the truck driver via a speaker system. The driver receives a printed pass that tells him where to park so that a massive yard crane can haul his container from the truck’s chassis to a stack in the cargo yard, where it’s loaded onto a container ship and floated across an ocean—or that entire process in reverse order.
On the morning of June 27, Pablo Fernández was expecting dozens of trucks’ worth of cargo to be shipped out from Elizabeth to a port in the Middle East. Fernández is a so-called freight forwarder—a middleman whom cargo owners pay to make sure their property arrives safely at a destination halfway around the world.*
At around 9:00 a.m. New Jersey time, Fernández’s phone started buzzing with a succession of screaming calls from angry cargo owners. All of them had just heard from truck drivers that their vehicles were stuck outside Maersk’s Elizabeth terminal. “People were jumping up and down,” Fernández says. “They couldn’t get their containers in and out of the gate.”
That gate, a choke point to Maersk’s entire New Jersey terminal operation, was dead, along with the rest of Maersk’s entire NotPetya-ravaged network. The gate clerks had gone silent.
Soon, hundreds of eighteen-wheelers were backed up in a line that stretched for miles outside the terminal. One employee at another company’s nearby terminal at the same New Jersey port watched the trucks collect, bumper to bumper, farther than he could see. He’d seen gate systems go down for stretches of fifteen minutes or half an hour before. But after a few hours, still with no word from Maersk, the Port Authority put out an alert that the company’s Elizabeth terminal would be closed for the rest of the day. “That’s when we started to realize,” the nearby terminal’s staffer remembers, “this was an attack.” Police began to approach drivers in their cabs, telling them to turn their massive loads around and clear out.
Fernández and countless other frantic Maersk customers faced a set of bleak options: They could try to get their precious cargo onto other ships at premium, last-minute rates, often traveling the equivalent of standby. Or, if their cargo was part of a tight supply chain, like components for a factory, Maersk’s outage could mean shelling out for exorbitant air freight delivery or risk stalling manufacturing processes, where a single day of downtime costs hundreds of thousands of dollars. Many of the containers, known as reefers, were electrified and full of perishable goods that required refrigeration. They’d have to be plugged in somewhere or their contents would rot.
Fernández had to scramble to find a New Jersey warehouse where he could stash his customers’ cargo while he waited for word from Maersk. During the entire first day, he says, he received only one official email, which read like “gibberish,” from a frazzled Maersk staffer’s Gmail account, offering no real explanation of the mounting crisis. The company’s central booking website, Maerskline.com, was down, and no one at the company wa
s picking up the phone. Some of the containers he’d sent on Maersk’s ships that day would remain lost in cargo yards and ports around the world for the next three months. “Maersk was like a black hole,” Fernández remembers with a sigh. “It was just a clusterfuck.”
In fact, it was a clusterfuck of clusterfucks. The same scene was playing out at seventeen of Maersk’s seventy-six terminals, from Los Angeles to Algeciras, Spain, to Rotterdam in the Netherlands, to Mumbai. Gates were down. Cranes were frozen. Tens of thousands of trucks would be turned away from comatose terminals across the globe.
No new bookings could be made, essentially cutting off Maersk’s core source of shipping revenue. The computers on Maersk’s ships weren’t infected. But the terminals’ software, designed to receive the electronic data interchange files from those ships, which tell terminal operators the exact contents of their massive cargo holds, had been entirely wiped away. That left Maersk’s ports with no guide to perform the colossal Jenga game of loading and unloading their towering piles of containers.
For days to come, one of the world’s most complex and interconnected distributed machines, underpinning the circulatory system of the global economy itself, would remain broken. “It was clear this problem was of a magnitude never seen before in global transport,” one Maersk customer remembers. “In the history of shipping IT, no one has ever gone through such a monumental crisis.”
Sandworm Page 20