Burn-In

by P. W. Singer

  They entered the building and descended a half-flight of stairs, the interior dark from the lack of windows. Here, in contrast with the EOB, the ceilings were low and the floor was a double-thick red carpet that muffled their footsteps. Thick-framed art covered the walls, each a battle scene or a portrait of a famous American. It was meant to be refined and luxurious, to awe and relax at the same time. It actually just felt dated.

  “And here we are,” said Gibbon, stopping at another steel doorway. Guarding it was only a single Secret Service guard behind a wooden desk. No body armor. No bots. Just a gold badge on a white dress shirt. It was as if further security measures would only insult those already inside the inner sanctum.

  “Where?”

  “He’s waiting for you in the Situation Room,” said Gibbon. “Well, it’s actually not one room, it’s a few of them together of different shapes and sizes, all the way down to broom closets just for secure chats. But you’re expected in the primary. Of course, he gets the good one. It’s the only advice I’d give you. Whatever he wants, he gets.”

  The White House

  Washington, DC

  At her first step inside the room, Keegan’s stomach knotted up. It was not the Homeland Security advisor waiting for them after all. At the far end of a long U-shaped conference table stood Willow Shaw.

  The energy he gave off was visceral, as if there were a million small machines churning away inside him. Given Silicon Valley billionaires’ predilection for biohacking and performance enhancement, that was in fact possible.

  “Agent Keegan, I am Willow Shaw,” he said, introducing himself as part of the charade that the famous play, acting as though people don’t already know who they are. “It is a great pleasure to meet you.”

  Shaw stood about a half foot taller than Keegan, with close-cropped silvery hair. It made his age hard to place, as the color could be read simultaneously as a sign of experience or distinctiveness. As he walked clockwise around the table, he dragged neatly trimmed fingernails across the back of each chair as if counting down five—four—three—two—one. He then reached out to shake hands.

  Shaw’s grip was a vise, as strong as a gunnery sergeant’s. Keegan took it as the usual male bullshit of trying to show dominance at the start of a conversation. Yet when their hands met, she was sure Shaw had never swung a kettlebell in his life. His palm felt like it was new skin, exposed to the world for the first time that morning.

  This was a hacker’s body, thought Keegan, consciously designed to crack the defenses of most social and cultural norms.

  He looked over to TAMS. “I am pleased to meet your partner, too,” he said, extending his hand to shake with the robot as well. She’d not seen anyone do that with TAMS, but it responded as another human would and shook back. As the two hands squeezed, veins and tendons snaked along Shaw’s forearm. His biceps flexed, stretching the sleeves of his black nanosilk polo shirt. The goal was clearly to physically dominate any room he walked into. With the robot, though, it seemed more like an engineering test.

  “Tell me something about yourself,” said Shaw. She read it as another attempt at dominance that the rich and powerful usually played, opening up the conversation with a command to be followed, veiled as interest in getting to know the person. It all wasn’t needed, Keegan thought. The unorthodox meeting and the rapidity with which Shaw had compressed space and time to get Keegan and TAMS into this room showed his power over her in the pecking order, while his entire business model was built on knowing people.

  But, as quickly as Shaw said it, he stepped even closer to TAMS. It seemed that the machine was the more interesting person in the room to him.

  TAMS stood still, not replying, even as Shaw moved closer to a distance that would make any human recoil at the invasion of their personal space.

  “Go ahead, TAMS,” Keegan said. “Authorized to answer questions from Mr. Shaw. Tell him something about your status.”

  “I am due to charge my batteries in approximately 139 minutes,” said TAMS.

  “Pertinent,” Shaw replied. “But it doesn’t reveal much of anything. How is Agent Keegan doing today?”

  Keegan again felt her stomach clench. Toying with technology and people came easily to Shaw.

  “Agent Keegan is currently showing above baseline levels of cortisol and adrenaline, with suboptimal levels of glucose. This can be attributed to her foot pursuit and subsequent apprehension of an armed robbery suspect and the chili and half-smoke hot dog that she ate,” TAMS said.

  Shaw laughed. It was a dry, nasal snicker that was out of character with his pseudo-dominant physical bearing. “I believe you just made a joke. Actually, I realize I didn’t need to laugh to validate it for you, which in and of itself could be construed as humorous,” said Shaw.

  Keegan squinted, her body’s natural reaction as her mind raced to try to figure out what the fuck was going on.

  TAMS stood silent.

  Shaw nudged one of the ergonomic desk chairs back and sat atop the table’s oak surface, as if he were in his own boardroom. “You and me, TAMS, are more alike than different. You are a machine with a mission, not just some servile knee-high domo fetching food according to an algorithm’s whims.”

  He bounded back up from the table and approached TAMS again, examining the robot’s head carefully. Keegan took a step back, getting out of the way, realizing Shaw didn’t even notice her in that moment.

  “You can read me,” said Shaw. “Tell me how.”4

  “I use multiple methods, with a combination of optical and LIDAR sensors,” TAMS replied. “Registration of body posture and facial micro-expressions. Transdermal electrical conductivity in determining physical response to external emotional stimulus.”5

  “All very good methods,” Shaw said, returning to his casual perch on the conference table as if considering which of the dozens of questions welling up inside him he would ask next.

  Keegan rocked back on her heels and clasped her hands behind her back; she’d determined the best course of action was just to wait to speak until spoken to and otherwise stay out of the way.

  “You’re learning all the time, aren’t you? Processing, sharing what you know with the other TAMS units in service in other cities,” Shaw said. “The network greater than the parts, them learning from you and your experiences with Agent Keegan here, and you learning from their experiences with their own Agent Keegans. The whole of the system then developing and refining new cognitive pathways like . . . Of course, it could go much better if we could have sent out something more cutting edge than yourself. Wet nets can easily surpass your processing power . . .” He paused.

  “I apologize, TAMS.” He stood again and made a perfect bow of regret in the style of a Japanese courtier. “That was impolite of me and, you’ll see as you get to know me, all too common. You and I are more alike than, say, Agent Keegan, here. You’re on a journey, an eternal one even, and the challenges you face in comprehending the world around you are so familiar to me. I envy you, though, and the speed with which you will make progress and evolve. My path is more arduous, painful even. I see that as an unnecessary aspect of the human condition—suffering. It need not be that way. In fact, nothing is necessary.”

  Shaw sat back down and faced Keegan, making eye contact to signal that now she was being spoken to. “My apologies also to you. My sensors and processing capabilities, if you will, are not like other people’s. They are in a way more like TAMS’s. People like me were once referred to as ‘little professors,’ because our hyper-intelligence manifests at an early age, as does our desire to share our enthusiasm for subjects dear to us.6 But, of course, it was not that. The condition is more akin to a difference in programming, which, in turn, requires one to reprogram yourself, to learn to read fellow humans.”

  Keegan wasn’t sure how she was expected to respond, so she did what Noritz had advised. “Yes, sir.”

  “That statement makes you uncomfortable, Agent Keegan,” said Shaw. “That is quite sim
ple to read. But there is no reason at all to feel sorry for me.” He stopped briefly, as if processing something, and then his mouth pursed into a slight smile. “Never feel sorry for someone who owns their own plane.”7

  “Mamet,” Keegan said back.

  Shaw’s face opened up, his eyes, cheeks, and mouth all moving in unison to create a wider smile.

  This is some multilevel chess shit, she thought.

  Was Shaw smiling in acknowledgment of an equal, someone who knew and perhaps enjoyed the same writers? Or had he just woven in the quote from Mamet as a subtle signal that he could track down anything he wanted about what she liked and knew? And was that signal a warning or some kind of strange flirtation?

  As if reading her, Shaw continued. “This is who I am. If I were made any other way, would I be a billionaire? Would I be in the White House, standing here with you? Of course not. God makes no mistakes; only bad human programmers do.”

  Keegan managed a nod and wished she’d insisted that Noritz come.

  “My education in reading people accurately has a functional end. It is not that I reorient my sensors and focus on the details, detecting the cues and tells that most people throw away as so much noise. It’s more. It also enables what you might see as ‘hacking’ emotions, turning those tiny observations into moments of exploitation.”

  “Like in poker,” Keegan said.8

  “For the highest stakes.”

  He turned back to the robot. “Let us run a comparison. TAMS, what do you observe about Agent Keegan’s biometric baseline, say, her micro-muscle and facial movements specifically?”

  “Rapid movement of muscles around the right eye indicates emotional discomfort,” said TAMS.

  “And the clenched jaw, more so just now—see it?—attests to a certain kind of frustration,” said Shaw.

  “Yes.”

  “TAMS, you must also register the weight of her body tilting back slightly on her heels, hands held behind her. It’s the posture of duty, intended to signal deference, but also, context dependent, can be a means to hide frustration. I am not able to get under her skin, at least literally, like you can. But it is enough to tell.”

  Keegan also had to pee and wondered what the two would make of that.

  “So why don’t we relieve her frustration?” Shaw said with a flourish. “What would you like to ask me, Agent Keegan? Anything?”

  “Yes, sir . . . I’d like to know what we can do for you,” said Keegan.

  “Nothing and everything. I was in town for a meeting, so it’s something like providence that puts us together today . . . Your incident at the restaurant caught my attention. And I was curious. This remarkable machine, and you, interest me. The president’s Homeland Security staff was kind enough to make it all possible.”

  “Providence,” Keegan said.

  “You seem suspicious of me,” he replied. “What do you think, TAMS?”

  “The data confirms it,” said TAMS.

  You’re damn right. You’re acting like you own the most important room in the White House.

  “I’m hungry,” said Shaw, more to himself than to either her or TAMS.

  He tapped on his left forearm, running his fingers in a circuitous pattern across the skin, activating a dermal interface.9 Keegan had never seen the rare and expensive technology in the flesh and she stared, marveling at the machine fused directly into the body.

  A moment later, Gibbon opened the door. “The White House steward system will deliver your order shortly.”

  “I’d prefer you do it yourself,” Shaw said.

  Keegan wondered if Shaw even understood the ask of having a human deliver food, let alone a White House policy staffer. Gibbon, though, nodded and ducked out.

  “While we wait, TAMS, why don’t you read me? Agent Keegan, what more do you want to know?”

  There was a lot that Keegan wanted to ask him, from where she could buy the superskin cream he was using to erase his wrinkles to what stocks to invest her retirement portfolio in.

  “What are you doing in Washington?”

  “I’m here all the time, actually. Care and feeding. House. Senate. Commerce. K Street. Sometimes CIA and DOD. Depends on the season. This trip was supposed to be a short one, but now I’m not sure that I’m going to rush back to California.”

  “TAMS, is he speaking truthfully?” She turned to the robot, trying to catch him off guard.

  “No,” TAMS replied. “External biodata and facial responses indicate Mr. Shaw is not telling the truth.”

  “Wonderful,” Shaw said. He cocked his head. “TAMS, try again now: I’m not sure I’m going to rush back to California.” He tipped his head slightly to Keegan.

  “TAMS, is he speaking truthfully?”

  “Yes,” the robot observed. “Mr. Shaw’s facial and other observable biometrics, such as facial muscle micro-movements and his pulse, all corroborate his truthfulness.”10

  “You sure?” said Keegan. It still didn’t feel right.

  “Sure as any machine can be,” said Shaw. “And just how sure is that, Agent Keegan? Do you think I can’t manage my own body? My whole life I have been learning how to present the kinds of data—tells, really—to make people more comfortable. But what is important is that you still detected the lie. No matter how good a machine’s sensors and processing are, they lack an intuitive understanding of the underlying data. That is the crux of the problem, isn’t it? It’s no different than what TAMS needs to understand if it’s going to be a successful partner for you.”

  “Mr. Shaw, maybe you should be teamed with TAMS,” Keegan said.

  “Now your humor is on display, Agent Keegan. Well done. But tell me, how is it really going with this unit? As you have detected, I am far from an impartial observer here. As a student of both technology and the human experience, I find utterly fascinating this collision between an evolutionary species and a revolutionary technology. But I also worry about what might happen next. I worry a lot about it, and that, Agent Keegan, is what I am really doing here in Washington.”

  “What do you mean by that, sir?”

  “Just as TAMS and I are similar, so are you and I, Agent Keegan. We are both warriors of a sort. For you it was the alleyways of East Africa and the Arabian Peninsula; for me, the no-less-dangerous hallways of this city. Different battles, of course, but which ones will determine the fate of this nation?”

  Keegan bristled at him equating running missions surveilling hacker dens in Mogadishu with dining around town with lobbyists in $5,000 jeans.

  “They may want to see it as an experiment, but let there be no mistake: you are now part of a battle that must be won,” he said. “This is bigger than you or TAMS, your Bureau, or even this government.”

  “Yes, sir.”

  “I don’t need TAMS to detect your doubt, Agent Keegan . . . May I call you Lara? As we have spoken long enough to cross that barrier, and we are, indeed, on the same side?”

  “Sure, Willow,” she said, not getting his permission for the reciprocity.

  “Lara, technology is essential to what makes us human in the first place, what distinguishes us from every other species, from the first crude tools our ancestors used to kill their food to the fire they used to cook it. And it has thus always filled our dreams. Indeed, it is why the ancients conceived of a technology like TAMS before humans even mastered the printed word, let alone mechanics or computing.

  “The Talos of Greek mythology, the ka of the Egyptians, the ‘precious metal people’ of Buddhist scholar Daoxuan, the golem of the early Jews—even as we moved forward to a world of reason, these technologies were what we aspired to. In the book Politika, so foundational to our modern ideas of politics that we take the very name from it, the philosopher Aristotle told how mechanical servants—robots—would be the one thing that would solve human conflict and inequality, making us the equivalents to the ‘gods on Olympus.’”11

  “But it didn’t work out that way,” Keegan said, knowing Shaw was eventually goi
ng to get to the “but” part of the story and trying to steer him toward it quicker.

  “Precisely. Aristotle writes at the same time as the myth of Pandora’s Box. By its novel nature, any new invention of significance must be destructive to society. And we innately know that too, as much as you don’t need to think consciously to process emotions.”

  “Anger is as much a tool as any created technology,” Keegan said.

  “Today’s fear of robots or algorithms taking jobs is the latest expression of something quite old, as are the demands from the streets for leaders to stop such transformation,” Shaw said, running his hand across his forearm. “Five centuries ago, William Lee applied for a patent for an automated knitting contraption and Queen Elizabeth I herself denied it, saying, ‘I have too much regard for the poor women and unprotected young maidens who obtain their daily bread by knitting to forward an invention which, by depriving them of employment, would reduce them to starvation.’”12

  The way that quote rolled so easily off Shaw’s tongue after his screen swipe made Keegan wonder if he was using implants. With his resources, he probably wasn’t on the cogniceutical drugs the US military experimented with for quicker training cycles. While they boosted focus and memory 10 percent, they also screwed with your sense of certainty, giving you insane levels of overconfidence, including for subjects you hadn’t studied.13 More likely, he also had the embedded brain interfaces Facebook had pioneered, back when it hired away dozens of DARPA scientists to work in its Building 8 skunkworks.14

  “Government cannot stop the future,” Shaw said emphatically.

  Keegan made a note to search that line afterward, guessing it had probably been audience-tested at a TED talk and then packaged for online distribution in a short clip. But Shaw very much seemed to believe it, if his expressions could be believed.