Rescind Order

by Natasha Bajema

  “Just tell me.”

  Grayson looked around him again before leaning in and said, “There appears to be a serious counterintelligence threat which has been linked to someone on your National Security Council staff.”

  “Who is it?” Susan asked, a lump forming in her throat.

  “Her name is Morgan Shaw.”

  7

  False Positive

  GRACE

  0600

  National Military Command Center

  The Pentagon

  Arlington, Virginia

  Grace stared intently at her computer screen, going over the input data and system outputs one more time. Each time she did so, the disturbing conclusion came back the same. She blinked several times, but she’d read it correctly. Then she blew out a few short breaths in an attempt to calm herself—but to no avail.

  This can’t be happening.

  Grace rubbed her sweaty hands on her pants and stared at the screen again. After the influx of new data inputs earlier that morning, ARC appeared to register an emerging nuclear threat from China. Consequently, it recommended an increase in alert status of U.S. nuclear forces to Defense Condition 3. But when Grace examined the input data, no such threat existed—at least not based on her assumptions about nuclear deterrence. That meant the ARC system was suffering from a fatal flaw that needed to be fixed immediately.

  Wringing her hands, she contemplated the implications of ARC’s recommendation. The Joint Staff’s Defense Readiness Condition (DEFCON) system consisted of different levels of alert and combat readiness for the U.S. military, ranging from the lowest peacetime level of DEFCON 5 to the wartime level of DEFCON 1. Since the Nightfall Incident, the U.S. military had remained at a heightened peacetime level of DEFCON 4.

  A move to DEFCON 3 would place U.S. nuclear forces on immediate standby for a launch order—something that had only occurred one or two times in all of U.S. history and not once since the end of the Cold War. If senior military leaders accepted the recommendation made by the ARC system and took the necessary actions, the United States would risk unnecessary escalation with China. And if the Chinese perceived U.S. actions as aggressive, its government would respond in kind. Grace didn’t even want to imagine what ARC’s next recommendation might be after the Chinese response. It might likely be a prelude to nuclear war.

  No one wants to think about that scenario.

  She sensed something was going terribly wrong with the intelligence component of ARC’s deep neural network, and she needed to warn the Pentagon leadership somehow. The main problem was she didn’t really understand what was going on and why. She needed more time. Grace glanced at her watch and shuddered. Only three hours remained before the pre-brief with the chairman of the Joint Chiefs of Staff in his conference room. She stood up and peeked over the top of her cubicle wall.

  Her large office suite located deep inside the National Military Command Center was still empty. She sank back into her chair and stared numbly at the screen. In less than an hour, Grace’s colleagues would start arriving at the Pentagon’s huge Top Secret facility which was located several floors underground. As the primary nerve center for the Department of Defense designed to monitor worldwide events and military activities in real time, it was protected against the effects of an electromagnetic pulse but could not withstand a direct nuclear attack.

  Before the ARC system became fully operational, the facility had supported the National Command Authority’s mission to rapidly launch a nuclear attack. A small safe a few cubicles over from Grace’s terminal used to contain the key for issuing Emergency Action Messages to the intercontinental ballistic missile silos, bomber squadrons, and nuclear ballistic missile submarines.

  The facility now housed the ARC system, which automated most of the functions needed to start and fight a nuclear war. To ensure continuous operation in the event of a nuclear attack, a network of nodes for the ARC system were scattered around the country and linked to the National Military Command Center.

  A salt-and-pepper-haired head popped over the side of her cubicle and moved along the edge like a prowling shark, causing her to jump slightly. Seconds later, Colonel Martinez entered her cubicle, a triumphant grin on his face as he straightened up his body. The deep crow’s feet around his eyes when he smiled made him look approachable despite his stature.

  Having worked for nine months in the Pentagon, Grace had come across many senior military officers. Martinez was by far her favorite, but sometimes he treated her like a little sister, and it got on her nerves.

  How can he goof off at a time like this?

  “Seriously?” Grace said, glaring at him. “You’ve got to know how focused I am this morning. You can’t possibly think you’ve scored another win just because you snuck up on me while I’m staring at this data. It could mean the end of the world and life as we know it.”

  “Always so melodramatic. A win is a win,” Martinez said with a smirk. Then his face became dead serious. “In your email, you said you had something important for the chairman’s read-ahead this morning?”

  “Yeah, it’s something, all right. But I didn’t want to interrupt your breakfast with Morgan,” Grace said, suppressing a frown.

  A twinge of sadness shot through her body when she thought about her waning friendship with Morgan. The thought of Morgan avoiding her at the White House meeting the previous day stung hard. It had been six months since she’d actually spoken to her good friend from her days in graduate school at Harvard.

  The U.S. Air Force paid for a few slots each year at the prestigious university, both to educate its top officers and to expose them to some of the smartest civilians in the country. Grace had been lucky enough to win one of the coveted spots to earn her master’s degree. Morgan had approached Grace during orientation, and they became fast friends.

  But ever since Morgan took the job in the White House, it was difficult to get an email reply, let alone a phone call. Since Grace worked twelve-hour days at the Pentagon on a regular basis, she was familiar with the burden of a high-level job. But she was still able to make time for the people in her life. And she could have used the extra shoulder in the wake of her father’s death.

  I’m sure reporting to the president takes things to a whole new level.

  “Nah, she’s cool,” Martinez said. “By the way, I told Morgan to give you a call since she had some questions about the ARC system I couldn’t answer. I told her you’re the best person to explain it. Plus, she owes you a phone call by now, don’t you think?” He gave her a knowing wink.

  Great, now she knows I complained.

  Grace replied sarcastically, “Gee, thanks.”

  “Okay, tell me about the problem with ARC,” Martinez said, plopping down in the extra chair in her cubicle, leaning back, and putting his arms behind his head. “Did you identify a bug in ARC’s programming?”

  “Nope, it’s not a bug,” Grace said with confidence.

  Martinez scrunched his face. “What about a cyberattack?”

  Grace shook her head. “I checked for that first thing this morning. As you know, deep neural networks are vulnerable to spoofing or data poisoning. That’s why the Department of Defense employs state-of-the-art, AI-enabled cyber defenses and firewalls to protect the data within the ARC system. They are impenetrable to outside forces. Even if a sophisticated hacker managed to break through, we would detect the breach. In the worse-case scenario, we could take ARC offline and revert to a previous version of the software.”

  “Okay, if the problem with ARC isn’t due to a bug or a cyberattack, what’s wrong?” Martinez asked.

  “ARC may have learned something we don’t want it to,” Grace said.

  Martinez put his hand on his forehead. “I don’t get it.”

  Grace took a deep breath. “Unlike a traditional computer program, ARC can learn from its environment. Each month, we expose the system to new infusions of ISR data. At that point, ARC tweaks its own algorithms to produce desired outcomes.�
��

  “But how does ARC learn the wrong thing?” Martinez asked, frowning.

  Grace gave him an uncomfortable smile. She desperately needed someone to listen to her and didn’t have time to go through her chain of command.

  Getting her points across to senior echelons in the Pentagon was hard enough as a major in the Air Force, let alone a young Korean-American woman. She often had to explain to people that she had not been naturalized, but rather was born in Korea as an American citizen. Her father was an American and served as a pilot in the Air Force near Seoul at the time of her birth, conveying her automatic U.S. citizenship despite her Korean mother.

  Somehow, this was difficult for some folks to understand, and they often assumed she was Korean by birth and spoke English as a second language. No one would ever admit to discrimination based on ethnicity and gender, but Grace often witnessed her white male counterparts getting their messages across more easily than she did.

  As the executive assistant to the chairman, Martinez could make sure Pentagon senior leadership heeded what she had to say. Although she’d known Martinez long before he started dating Morgan, this was the first time she’d decided to lean on her personal connection in her professional career.

  It was a big risk to go over her boss’s head; it could cost her job. But she’d tried for months now to convince her leadership that there was something not quite right with the ARC system, and they wouldn’t listen.

  At least now, she had some tangible evidence of its problems, and she didn’t plan on wasting the opportunity.

  Grace grimaced. “Well, it’s a bit complicated.”

  “Maybe you can break it down into small pieces first, and we can figure it out together?”

  Grace’s stomach tightened into knots. Here goes nothing.

  8

  Demonstration Blues

  DREW

  0610

  Lafayette Square

  1600 Pennsylvania Ave NW

  Washington, D.C.

  Drew Hudson rubbed his eyes and yawned as he peeked out from the huge golf umbrella covered by a giant blue tarp. He’d already slept as long as he could on the paper-thin sleeping pad—no more than four hours. He stretched his achy arms and shoulders. At one point in the middle of the night, he’d rolled onto the warm cement and woken up with a rough texture on the side of his cheek. He touched his face. Thankfully, his skin was smooth again.

  Holding the edge of the tarp, he stepped out into the open and took a deep breath, nearly choking on the dust suspended in the air. Drew covered his mouth with his arm, sneezed, and then shook his head at the irony.

  Maybe climate change will kill us before robots get the chance.

  Despite the early hour, the intense summer humidity meant being outdoors felt particularly uncomfortable. The record-breaking temperatures and high moisture in the air produced more smog than usual, making it dangerous to one’s health. But none of it would dampen the spirits of the protesters who’d come out for the Campaign Against Killer Robots.

  If we don’t speak out now, humanity is doomed to its own devices.

  Drew wiped sweat from his forehead and surveyed the temporary camp. The hodgepodge collection of tents, umbrellas, cardboard boxes, and lawn chairs stretched the entire length of Pennsylvania Avenue, located just outside the White House. There must have been at least a couple hundred diehards who had camped overnight for the 24-hour vigil, sleeping in the streets. Many more would arrive throughout the day to make their final stand against the Department of Defense’s plan to field fully autonomous weapon systems.

  Many of the protesters were already chanting, marching back and forth, and waving their signs. They were desperate to sway President Tolley’s decision to veto the legislation that would authorize the transfer of kill decisions from humans to machines. The protesters had a special name for the bill—they were calling it the Killer Robot Directive. They shouted over and over: “Ban killer robots, veto the bill! Ban killer robots, veto the bill! Ban killer robots, veto the bill!”

  Drew stuffed his hands in his pockets and tried to ignore the low growl in his stomach. Though determined to prevent the development of the Terminator by the U.S. government, he was definitely not a morning person.

  It’s way too early for this.

  He would never have volunteered himself to sleep in the streets. Except he’d accidentally mentioned to his friends that he wasn’t planning on drinking any alcohol at the previous night’s party. So he was the one who got stuck with the night shift. In order to camp outside the White House, someone needed to attend each individual spot at all times. Otherwise Secret Service would dismantle it for security reasons.

  The night shift turned out to be more convenient, anyway. Drew was taking a summer class at Georgetown first thing in the morning, and he refused to skip it for anything. After all, he was taking on massive debt to pay his way through grad school. He’d done the math on the hourly cost of his education—it was obscene. His student loans didn’t come even close to covering his living expenses.

  He pulled out his smartphone, checked the weather, and chuckled to himself. In just a few hours, the sun would blaze down on the makeshift tent, and actual temperatures would reach over a hundred Fahrenheit. He didn’t want to think about the heat index. Maybe the night shift was better all around.

  Stretching his shoulders and cracking his neck, Drew unfolded a lawn chair, plopped into it, and reached for his thermos on the ground. The coffee from the previous night was room temperature, but he gulped it down anyway, desperate for a caffeine fix.

  Then he glanced at the protest signs leaning against an empty suitcase. For a moment, Drew considered whether he should add his voice to the chant. The louder the protesters were, the better chance they had for the president to hear their shouting inside the White House. That’s what they told him, at least.

  But do we have a shot at influencing Tolley’s decision?

  Before he had a chance to get up, a gray-haired lady with bronze, leathery skin walked toward his umbrella. She was pushing a cart of groceries in the direction of a large tent covered in anti-nuclear signs at the end of his row. Her mismatched clothes were dirty and torn in places, and her curly hair was unkempt. A collection of anti-nuclear and world peace buttons adorned her jacket. As she started to pass him by, the smell of cheap liquor wafted in his direction, causing him to pull up his nose.

  As if sensing his disgust, the old lady turned to give him an icy glare. “Hey kid, what’s your problem?” Her voice was unexpectedly deep—gravelly, as if she’d spent many years smoking cigarettes.

  Or from breathing in this smog every day.

  He shrugged and gave her a half smile. “Sorry, ma’am. No problem here,” Drew said as politely as he could. He didn’t want to get into it with anyone, let alone a drunken, homeless lady.

  “You guys are way late to this party, you know,” the lady said snidely, waving her hand around at his fellow protesters. “In more than one way.”

  “What do you mean?” Drew asked.

  “Well, we’ve been out here for decades, doing our part to save the world from itself. I’ve spent the past twenty years camped out here, calling for an end to nuclear war.”

  Twenty years?

  “Who’s this we?” Drew asked, furrowing his brow.

  “Oh, you haven’t heard of us? The White House Peace Vigil?” When he didn’t respond, she said sharply, “Where have you been living? In a cave?”

  He shrugged. “I don’t know that much about nukes. I’m more concerned about risks of artificial intelligence.”

  An annoyed look formed on her face. “The nukes will kill us first, you know. But I guess that fancy new technology is more interesting for you young people. Anyway, we’re the anti-nuclear activists who have protested the existence of nuclear weapons, warning anyone who will listen about the coming end of humanity. And guess what?” Her voice slurred slightly.

  “What?” Drew asked.

  “Nuclear
weapons continue to threaten humanity today.” She grinned sardonically, showing several missing teeth. “Maybe there are a few less weapons, but they still exist. And it only takes one to destroy an entire city. The people in power say we can’t put the genie back in the bottle.”

  Drew quirked an eyebrow. “What’s your point?”

  “Are you prepared to wait that long for change, young man?” the lady asked. Before he had a chance to answer, she said sarcastically, “Because I doubt it. And we humans seem to have to learn from our mistakes first… that means we’ll field autonomous weapons systems before we understand the danger.”

  Drew fidgeted with his hands. “We’re not planning to be out here forever. Just hoping to convince the president to veto the autonomous weapons bill today.”

  “Ha.” The lady’s head fell back in a hearty cackle, which turned into a ripple and then a shake. She grabbed her stomach and began trembling from laughter. After a few moments, she recovered and said, “You young kids are all the same. Naive about the ways of the world. Impatient to affect change. Unwilling to make hard sacrifices. Your little autonomous ship has already sailed off on its own. Today’s legislation will simply give the Pentagon the green light. If you want to reverse what’s already happened, you’re going to have to do way more than protest a bill that flips a switch.”

  Drew frowned, worrying she might be right. “Well, if you think we can’t achieve anything by protesting, why are you still out here?”

  She stumbled a bit, and her eyes dimmed in her stupor. “I guess because I stayed too long. I was young and naïve when I first started… much like you. I didn’t have a penny in my pocket or know what else to do to make my voice heard. When a few months turned into years, I wasn’t sure what to do differently.” She looked him up and down, her lip curling. “I wasn’t a rich kid like you, boy.”

  “Well, I’m not rich,” he said with a hint of defensiveness.