Burn-In

by P. W. Singer

  The old man visibly stiffened at the resistance. So much so that Preston backed up a half step. Then, the old man softened his expression into a smile.

  “I bet there’s never a good time with somebody like you, and I know just what that’s like, Professor. I promise it’s only a moment if you can indulge me. I’m keenly interested in your Automated Decision System work and how the open-source model of Linux influenced it. Many may not remember it, but for me, it was life-changing stuff.”

  “Oh . . . You do know my weakness. How can I turn away an alum who wants to talk ADS and Linux?” said Preston. “A few minutes then. Come in and don’t mind the mess.”

  Preston turned his narrow shoulders and beckoned the old man inside. “Mess” was just another affectation. One freshly painted office wall featured an oversized photo of Preston hiking in the Himalayas, a lower body exoskeleton wrapping his hips and knees, easing the load—technology allowing man to conquer new frontiers without the accompanying sacrifices that make them worthwhile. A series of smaller pictures showed him meeting the US president, the UN secretary general, and technology industry luminaries. The old man noticed a faint hairline crack in the wall’s plaster that strung the photos together like a crude link analysis display. The office smelled freshly cleaned, no stale food or unwashed socks anywhere. A projected screen displayed a representation of the VR model he was running. It looked like a mosaic of earth-colored tiles arranging themselves by pairs. Preston reached into the projection and waved his hand sideways, as if brushing crumbs off a table, and the projection disappeared. Then, he jauntily pushed the glasses up on the top of his head before sitting, his gaze softening.

  “Ramen and Red Bull. There would be no ADS without those two gifts from the heavens. I was a grad student at MIT, in Washington for a summer, supposedly working for NASA when they froze all R and D funding in that budget fight. Remember when they stopped paying the debt for a couple months? It was then, so I was broke and starving—literally. Then another student who was working on DC Metro’s driverless cars let me know they had some extra funding that he could get me. It wasn’t much, but it kept me in calories and code.”

  “And that all led to this place,” said the old man.

  “Exactly,” said Preston. He sighed, seemingly lost in his memories. “If I had never worked on that old ADS software, I never would have gotten hired at my first faculty job.”

  “And my wife and son would still be alive today,” said the old man.

  Preston looked at him quizzically, still not understanding what he had done years ago, even as the brass top of the cane smashed down on his head. The strike split the VR glasses in two, sending them clattering to the floor, and a jet of blood splashed across the wall above the workstation, leaving a crimson arrow pointing skyward. Another blow. And another. The old man didn’t stop until the cane surrendered to his rage with a crack, the brass ball falling to the floor.

  The old man stood over the body, looking down at what he had done. The body looked different than how he’d long imagined—no look of fear or accusation on Preston’s face. Just the mess that came after something that had to be done. It was then that he realized he wouldn’t even need the sterile wipes he’d brought just in case he threw up.

  He kicked Preston’s chair out of the way and placed a small black rectangle next to the computer under the desk. While the drive downloaded the contents of Preston’s computer, the man quickly stripped off his blood-spattered blue dress shirt and khaki slacks. He then peeled off the 3-D printed silicone mask.4 It had been designed using an algorithm that generated a hyper-realistic face an AI had dreamed up by blending the features of famous celebrities.5

  The contact lenses came out with more difficulty, bringing forth a rush of tears. It was not mourning over the taking of a life, just the small price of something that had to be done.

  A generation’s worth of age disappeared as he changed into a pair of tight gray jeans and a puffy down black nylon pullover. He changed shoes, into a pair of basketball sneakers, but with lifts on the left side, which would alter his step and throw off any gait recognition software.6 Then he went to work on the facial recognition side, putting on a pair of thick, glossy black Nike AR glasses, and slipped in a set of prosthetic teeth with an overbite. A small foil pouch held a moist wipe, which he ran over his cheeks, lips, and chin.7 The girl on the train’s anti-surveillance makeup was for fashion. People who weren’t just playing at rebellion used skin lotion with microscopic refracting beads. The beads were invisible to the eye, but they distorted camera imagery at the pixel level. It was the only way to be truly free in a world of algorithms designed to mark and track humans. It was apt that the algorithms themselves had originally been honed by prison inmates paid pennies to train AI as part of cheap human labor.8

  Footsteps in the hallway stopped in front of the door, and the man froze. He felt his chest tighten and tensed his stomach muscles against the sick feeling that welled up. Strange. He analyzed the feeling, something to be understood as much as fought. Killing Preston hadn’t caused it, he reasoned; the thought that he might have to do so again so soon was what made his stomach flutter.

  Don’t open that door, he willed whoever was on the other side. He knew that chance played a role in so much of life—whether you got a certain disease or struck by lightning even. But the deaths of his wife and son were different. There was code. There was causality. There was Preston. God’s hand was not present when their two bodies were violently merged into the 39 tons of metal of their DC Metro subway car. On the way to see him. At work. Writing code himself.

  A gentle knock on the wooden door, barely penetrating the oak. A young man’s timid voice. “Professor? It’s Marshall Winters. I’m early for office hours but thought you might be free to discuss my research project?”

  Silence.

  Another knock.

  The man kneeled carefully, quietly picking up the wooden cane. Frayed and jagged wood, like tiny blood-soaked teeth, where the brass top had been. A simple tool. Man’s original weapon. Would he have to use it again?

  Another minute passed, then a quiet bashful voice. “Professor Preston, if you’re there, I’ll come back at office hours . . . I’m sorry to bother you.” A defeated sigh, audible even through the thick oak door.

  The man waited another minute, counting down the time on his watch. It wasn’t one of the new Watchlets or even an old networked one. It was just a ruggedly simple steel automatic watch without even a date display. As the second hand smoothly swept around the black dial, he took a breath every five seconds to calm himself.

  Time to go.

  But when he gripped the door handle his hands shook uncontrollably. For a moment he could not remember where he was. It wasn’t the weight of what he had done but how he had gotten here. As he doubled over, blood rushing to his head, it was as clear a picture as if he had seen it on Preston’s VR kit. They were on their tree-lined street, his wife and he each holding their son’s hand. The boy hated touching the lines on the sidewalk, so they would swing him by the arms over them, turning his worried steps into leaps of glee. That day, a fire truck drove by, and his son stopped to salute. The driver turned on the lights, a playful flash of red, and gave his son a thumbs-up. He had been so excited, even worried that his parents might have missed the moment of a lifetime. “Did you see that, Daddy?”

  They had died the very next day.

  As the memory passed, his hands stilled. A final look around the room left him satisfied. This was justice. It was right. Preston and others like him had lived their lives according to a code, and it was broken. They celebrated the idea of “disruption,” because they didn’t see themselves as actually being responsible for the consequences of their actions.9 Now, that would change.

  He was about to open the door when he stopped again. There, in the corner. With a now steady hand, the man reached under Preston’s desk to retrieve the brass knob that had broken off the top of his cane. With sure fingers, he
wrapped it in a plastic glove and slipped it into the messenger bag alongside the hard drive, careful not to get any blood on his hands.

  FBI Domestic Special Detention Facility

  Reston, Virginia

  The robot was smaller than the average human. Three black metallic ropelike ridges ran along the top of the head, antennae that looked almost like cornrows. The silver-dollar-sized eyes were greenish blue—Keegan assumed it was programming that mimicked the eye color of the person looking at it—nestled in an ovoid basketball-sized metallic and ceramic head. A sharp-edged nose with nostrils, probably a design feature blending form and function for venting, divided the robot’s face at midpoint. There was no mouth, which made it look to Keegan like a Japanese Noh mask she’d seen at the Freer Gallery in the Smithsonian. Bad design, she thought. You could easily put another set of sensors there.

  The robot’s “skin” absorbed the harsh light thrown off by the fluorescent tubes overhead. It was a peculiar effect, a void placed in the center of the room. Faintly stippled, the gray material looked almost brittle, like an eggshell. Keegan guessed, though, the opposite was true. It was likely ceramic composite, lighter and stronger than steel. The hard, light gray material covered the top of its hands, but the fingers themselves were black, most likely some kind of rubber for gripping.

  Keegan was tilting her head to the side, to try to look under the table at the robot’s actuators, assuming it could walk, when she noticed that Reppley’s screams had quieted into something more like moans. She looked back at the man. Snot and saliva dripped off his chin, into his beard, and he was now rocking back and forth, trying to pull away from the table, but each time crying out when his hands were caught by the cuffs. Probably now feeling the pain kick in from nearly breaking his wrists wrestling with the shackles. There was also that distinct smell from the train station’s entrance again.

  “You’re a mess,” said Keegan as she stood up to leave. “Stay here. I’ll be back in a few to consider the situation at hand.”

  “Understood, Agent Keegan,” said the robot.

  “I’m not talking to you, bot,” said Keegan. “I’m talking to the human.”

  Outside, Noritz waited in the hallway with an apologetic expression, trying to head off Keegan’s certain explosion. “I swear I had no idea,” Noritz said. “After you went in, I got a ping that there would be an observer in the room and I was distinctly told not to notify you.”

  “That’s bullshit. And you know it,” Keegan replied. “Totally unacceptable. You do not hold back critical information, let alone introduce an element like that into an initial interrogation. You saw how he freaked out, like he thought the bot was going to torture him. When we’re in the room we put the pressure on a suspect, not machines. This isn’t Beijing, where you can just unleash a bot on a prisoner. What the hell is going wrong with everybody here?”

  “I understand you’re mad, but you’re taking it out on the wrong level of the food chain, Keegan,” Noritz said with deliberate calm. “Again, this was not my call. Fortunately, you can share your concerns with those who made it. They’re waiting for us in the conference room.” He paused and his bureaucratic veneer cracked a bit, the Pennsylvania state trooper coming back in. “I would advise, however, that you calm the fuck down before you go in.”

  Keegan followed Noritz down the hall to the conference room, stewing in silence as they walked. When they entered the room, a phalanx of senior agents and Justice Department officials awaited them at the table. Keegan knew a few by sight, the special agent in charge of the DC region, Noritz’s boss’s boss, as well as the chief of the national security division for the US Attorney’s Office for the District of Columbia, who had the jurisdiction on terrorism cases.10

  Each of them had VR goggles laid out in front of them, except a man in a dark blue suit at the head of the table, who was still wearing the goggles and giving ongoing color commentary. Apparently they’d all been watching the interrogation from the robot’s point of view.

  “Look at him still blubbering away. I don’t want to sound too old, but they don’t make terrorists like they used to.”

  His assistant tapped him on the shoulder and he took off the goggles—Keegan recognized it was the deputy director. Kamal Bosch held the number two position in the Bureau and, even more important, the highest rank that an FBI agent could reach without needing appointment by the president and a vote by the Senate. That made Bosch not just the lead on every major FBI issue and investigation, but also its institutional steward, the ultimate internal guardian of who and what the Bureau was supposed to be.

  Bosch motioned for Keegan and Noritz to sit at the table, with a hand gesture that also indicated the two senior agents there were to get up and move to the seats running along the wall for the backbenchers.

  “Aggressive work, Agent Keegan. Not just on the arrest today at Union Station, but in there with the suspect. Well done.” Bosch sounded sincere, but coming from such a senior official in a three-piece suit, it still left Keegan wary. Even more so when the suit he was wearing was real wool. Keegan also caught a whiff of his cologne, Confiance, imbued with pheromones that supposedly influenced people.11

  “You two were quite the . . . I don’t think I’ve ever seen a suspect broken down so rapidly like that.”

  “Thank you, sir. But—”

  “Keegan,” said Noritz, trying to cut his agent off before she went too far.

  “No,” said Bosch, running his hand over his razor-shorn scalp. “Let her continue. All of us here are in Agent Keegan’s debt today, in more ways than even she knows.”

  “Sir,” said Keegan. “With all due respect—”

  “That I will not allow, Agent Keegan,” said Bosch quickly. “I know what you mean by that. Just say what you want to say.”

  “I apologize, Deputy Director . . . but what just happened is total horseshit,” said Keegan.

  Noritz let out a slight grunt of pain beside her, but said nothing.

  “That’s better,” said Bosch, folding his arms as if in judgment. “And your assessment of this is why, Agent Keegan?”

  “Two reasons, sir. The first is that machine took over my interrogation.”

  “Except it didn’t. The system simply revealed key information in real time,” said a thin man three seats down from Keegan. Maybe late thirties or forties, tall, but his height underscored his seeming fragility. He wore a black suit like most of the others, but his was set off by cyan-colored hair that shifted from a green or blue shade, depending on the angle of the light. It was mesmerizing in a certain way, any slight movement of his head changing the coloration. It was also obvious peacocking, common among civilians, but still abnormal in the Bureau. The real tell, though, was how his hands made a triangle under his chin, as if he had been analyzing the conversation rather than an audience to it. It was an intended signal that said, while he had a seat at the table, he was of a different professional guild than the investigators and lawyers gathered around it. Profiler most likely.

  “Agent Keegan, since you didn’t let us get to introductions, meet Dr. Sander Modi of Behavioral Analysis Unit 5. . . who evidently disagrees with you,” said Bosch.

  So that answered two things. First, who he was. Unit 5 was the research wing of the Bureau’s National Center for the Analysis of Violent Crime (NCAVC).12 It had started out with the psychological profilers hunting serial killers back in the 1970s and then moved into anything and everything from all the sciences that might aid the Bureau.

  And, second, Bosch had just given away a key part of his own psychology. He was one of those leaders who liked to have the issue debated in front of them, portraying himself as the dispassionate judge, when he had likely already made a decision.

  “I’m sorry, sir, but all of you, including Dr. Modi, were not in the room. VR gives you the sense you’re there, but you’re not. It’s different in there. An interrogation turns on the emotional read you get on the subject. And that machine just completely threw the
subject’s emotional state out of whack. Everything we got from Reppley, we could have obtained through other means, without the robot . . . without the blowup.”

  “Your response, Dr. Modi?” Bosch asked, raising his eyebrows.

  “Actually, the ‘blowup’ was the ‘key information’ I meant. Agent Keegan certainly received the identifying information much more rapidly than otherwise possible and, importantly, in the active context of this inquiry, ‘in the room,’ as she puts it.”

  Keegan nodded in assent; he’d made a smart pivot off her main point.

  “But what is more notable,” he continued, “is what happened next. You’ll note the system detected a pheromone level that was well off the scale. The level of fear that the prisoner displayed was disproportionate in the extreme and certainly does not align with any expected profile models from the scenario of his arrest. Without the robot, we would not have induced that reaction.”

  “His reaction was that he went nuts. Aren’t they all?” said Keegan. But inside, she knew that was not the case. Frankly, the grown man’s meltdown bothered her more than the surprise appearance by the robot. She had seen adults crumble before, but it always happened from a trauma that struck like an emotional lightning bolt—hosing out an M-ATV after losing a buddy in battle, the video message that a mother had died while you were thousands of miles away. This was different; Reppley had just lost it at the simple idea of being locked in a room with a robot.

  “Agent Keegan, you know you’re grasping on this one,” said Bosch, now adjudicating. “He’s right, both on getting useful data in the moment and that a subject going ‘nuts’ is something pertinent. What is your second concern?”

  “My second concern is that, even if you were going to put a system in the room with me, I should have known. You can’t treat a live interrogation like some kind of experiment, putting me in there cold with some thing.”

  “Actually, I can, Agent Keegan. The whole point was the experiment. That thing is a learning machine, as are you,” said Bosch, “and as the events have proven today, a damn fine one. Both of you went in there ‘cold’—without any sense of what awaited—and yet teamed up to achieve a better result. What we want to learn next is whether that can be replicated. I think it’s best Agents Keegan and Noritz and I take a walk. Dr. Modi, will you bring TAMS to join us?”