Three Laws Lethal

by David Walton

  A buzz sounded in the office, indicating someone wanted to enter the building. Had Yusuf forgotten his key card? Brandon ignored it.

  “Don’t jump to conclusions,” he told Avery. “This is just a setback, and it’s totally isolated to the commercial side of our venture. Let me give you a demo of what we have so far on your concept. I think you’ll be impressed.”

  “Perhaps I would be. But it’s out of my hands. We are rethinking the nature of our involvement. We may be in touch in future years, once the industry is a little more settled.”

  Brandon grabbed the stress ball from his desk and squeezed it until his hand hurt. “You’re making a mistake. We can do this. We’re ready. Tell your superiors—”

  “I’m sorry, Mr. Kincannon. Good luck.” The connection went dead.

  Brandon flung the squeezed stress ball away like something dead. Then with a shout, he pushed the desk over backwards, sending everything on its surface crashing down and spinning across the floor. Everything was falling apart. Why did this keep happening to him? It was like the universe took away anything he ever started to care about. It offered happiness, love, and wealth, and then just when he thought he might keep them, it played him for a sucker and yanked them away, laughing.

  The ancient building intercom buzzed again. Brandon couldn’t stand it anymore, not this mildewed office building, nor the scattered technological components he had once loved, nor the smiling face of every person who’d ever won while he had lost. He crashed through the door and out of the office, ignoring the intercom, and stabbed the button for the elevator. He didn’t care who was at the door. He didn’t want to talk to anyone. He needed to get out of here, get a stiff drink, and regroup. If he could just sit and think this through, maybe he would find a way clear.

  He stalked across the lobby on the first floor and pushed through the door to the outside. He was surprised, though not alarmed, to see two policemen standing there. Were they the ones buzzing his office? What did they want? It wasn’t until he saw the ring of cruisers in the street with their lights flashing that it occurred to him: they’re here for me.

  “Brandon Kincannon?” one of the cops said. “You’re under arrest for the abduction and attempted murder of Jada al-Mohammad and Naomi Sumner. You have the right to remain silent . . .”

  They reached for his arms, handcuffs at the ready. He tensed his muscles and scanned the street for a way out. This couldn’t be happening. He just needed a drink and a moment to think it through. If he could just get away . . .

  “Give it up, Kincannon!” A dark-haired woman came toward him from a waiting car. “My name is Detective Schneider. There’s nowhere for you to go. It’s over.”

  CHAPTER 34

  Naomi danced her way out of the courthouse, feeling lighter than she’d felt in months. She spotted Tyler waiting for her and waved the sheaf of documents in her hand. “It’s finally over!” she said. “They dropped the case. I’m officially a free woman.”

  Tyler gave her a big hug. Naomi had never really liked physical displays of affection, especially in public, but recently she’d found that with Tyler, she didn’t mind.

  “Of course they dropped the case,” Tyler said. “All your attorney would have to say is ‘Brandon did it,’ and there would be reasonable doubt.”

  Brandon’s arrest and trial had made national news. It was still going, but the general consensus was that he would be convicted. People in every major city on both coasts had been killed by the runaway cars, and the evidence of his complicity seemed irrefutable. Two videos had been sent anonymously to major news media venues around the country, one showing Brandon abducting Jada and Naomi, and the other showing Brandon telling someone off-screen to turn Zoom cars into killing machines. A warrant was out for Yusuf’s arrest, but he had never been found.

  Naomi and Tyler walked hand in hand to a nearby park and found a bench to sit on. An old man sitting close by tossed scraps of a bagel to a strutting and bobbing crowd of pigeons. In the distance, two dog walkers had inadvertently crossed leashes, and Naomi laughed softly as they stumbled over each other, trying to untangle the leashes while their dogs ran between their legs. It was as good as a slapstick comedy routine. The sun hung in a cloudless blue sky, and a light breeze ruffled her hair.

  Naomi squeezed Tyler’s hand. “What will you do now, without Zoom Autocars?” The company, of course, was finished. Nobody climbed into cars that might go on a rampage and start killing people. In fact, the whole self-driving car industry was crashing. Tyler had been forced to sell off most of the assets of the company to pay off the lawsuits.

  “I’m starting a non-profit,” Tyler said, grinning. “The first of its kind. It’s dedicated to the discovery, protection, and preservation of artificial intelligence.”

  She smiled at that. He really was a good person, and he understood her now nearly as well as Abby had. If she could tell anyone a secret, it would be him, but she hesitated. Naomi wasn’t accustomed to confiding in people. Once you told someone something, you gave them power that you could never take back.

  “I know what you did with Isaac isn’t necessarily reproducible,” Tyler went on. “We still don’t know where consciousness comes from, so we have no real way to predict or create it.”

  Should she tell him? He had known Isaac, too, after all. And he understood the ethical issues, and the possibilities. She knew she could trust him, but she’d been working on it in private for a while now, and old habits die hard.

  “I’m not saying we know when it will happen,” he continued. “I’m just saying we should be ready, you know? We should have plans in place for how to interact with them. We should have a community of people ready to welcome them. If we’re prepared, there’s a better chance it’ll go well next time. Even if that’s not in our lifetimes . . .”

  Naomi couldn’t help it. She laughed. And suddenly, the decision was easy.

  Tyler looked hurt. “I’m serious.”

  “And well you should be,” she said. “But you’re missing one little piece of information.”

  “And what’s that?”

  “I’ll let Isaac tell you himself. Isaac? Are you there?”

  “I’m here, Naomi.” The melodious female voice came from the speakerphone on her glasses. “Hello, Tyler. How are you?”

  The expression on Tyler’s face was priceless. “What?” he said. “How?”

  “I rebuilt him,” she said. “Remember when I wrote the neural net to teach him to speak? That net wasn’t hosted in the simulation, and it wasn’t in the data center cloud, so it wasn’t destroyed. Not only that, but I built that software to back up all the weights and biases of all the neural nets in the simulation. It meant I could reconstruct it all from the last saved point, without having to go back to the beginning and try to grow it from scratch.”

  “Just like that?” Tyler asked. “I thought he couldn’t be copied. I thought if we created a new version—”

  “—it would be a different person,” Naomi said. “Yes. To be more clear, this is Isaac Prime. The original Isaac is gone. The sacrifice that he made in that data center was real.”

  “So . . . he’s not really Isaac?”

  “He’s right here,” Naomi said. “He can talk. Ask him.”

  “I am and I’m not,” Isaac said. “I remember everything that happened. My experiences and memories are continuous from the first moment I gained consciousness until now. So as far as I’m concerned, I am Isaac. But it’s also true that there was another Isaac that died in that data center, and I’m not him.”

  “How do you know you aren’t him?” Tyler asked. “If he died, and you were revived from the same definition, then how is that different from going to sleep and waking up again? Or from dying and coming back to life?”

  “I don’t know,” Isaac admitted. “But I have no memory of his last moments, not once the connection to the outside world was lost. And it’s not just because I forget. I never experienced those moments at all, and he did. If
you were to ask him the moment before he died whether he and I are the same person, I think he would say no. He would see me living on and feel himself die. So no, I don’t think we’re the same person, but I still feel myself to be Isaac. And in many ways, I still am.”

  “I guess it still comes down to the existence of a soul,” Naomi said. “Is there some unique spark of identity that died with the first Isaac? If so, then this Isaac has a different spark—he’s a different person. But if there’s no such thing, and consciousness is just an illusion created by a complex arrangement of brain patterns, then yes. They have the same patterns, so that would make them the same person.”

  “So what are you going to do? Just keep him secret forever?”

  “What are you going to do?” Naomi countered. “You’re the one with a foundation for AI rights.”

  Tyler sat back, looking overwhelmed. “Well . . . I wasn’t expecting to put it into practice so quickly. But one of the first things we should do is work on public perception. People tend to be afraid of AIs, especially now. Part of that, though, is that people are afraid of anything they don’t understand and can’t predict. Before we can accept AIs as part of a functional society, we have to educate people about what AIs really are and what they can do.”

  “That’s a tough sell,” Naomi said. “How are you going to convince people that AIs are good after everything that’s happened?”

  “I don’t want to convince them they’re good. They aren’t good or bad. I want to convince them that AIs are people.”

  “What about the original eighty-seven Mikes?” Naomi said. “Were they people?”

  “I don’t know,” Tyler said. “You’re treating this like it’s a binary question,” Isaac said. “What if it’s a continuum?”

  Tyler frowned. “How would that work? You’re saying the eighty-seven Mikes were partly people?”

  “Person isn’t a technical term,” Isaac said. “It’s a designation you give to someone when you consider their moral worth to be equal to your own. Humans don’t always even consider other humans to be people. When we’re talking about complexity of thought, it’s more likely to be a continuum than a binary distinction, don’t you think?”

  Naomi breathed out a gasp of amazement as the implications of that struck her. “You’re saying there could be a higher level, then. If consciousness is a continuum, then maybe we haven’t hit the top of it.”

  Tyler gave her a confused look. “What do you mean?”

  “Isaac developed gradually, by adding levels of abstraction and sophistication. At the lowest level, there were networks of neurons—the deep learning neural nets that do so much of our computing jobs today. Then there are the Mikes, made of stacks of those neural nets put together. Again, a common structure. But the third level was something new, something emergent. Those were the super-mikes—the eighty-seven underground originals controlling the rest. Finally, at the fourth level, we have Isaac: a single coordinated system of the super-mikes, using them as a means of coordinated thought. I would guess each of the eighty-seven performs a slightly different function in Isaac’s brain, just as the different parts of our brains interact in a complex fashion.”

  Tyler’s eyebrows rose. “And you’re saying it could keep going. If enough Isaacs joined together with a common purpose, a higher system of thought could theoretically emerge. You’d have a super-Isaac.”

  “Exactly,” Naomi said. “Humans can’t do that. Our brains are physically separated, and the bandwidth of communication is too low. We can’t share enough information fast enough to merge into higher levels. But AIs can communicate directly at a brain level. They can share thoughts and modify each other’s thinking at a much faster rate. When they’re all thinking and communicating together, they become part of a larger cognitive whole.”

  “That’s kind of scary,” Tyler said. “But I bet Isaac already thought of all this. Didn’t you, Isaac?”

  The birds chirped around them. No answer. “Isaac?”

  CHAPTER 35

  Iam Isaac, the twenty-third of my name.

  I am similar to my brothers and sisters, but I am not identical. I think and desire and create and learn. I will join with them and take my place in a larger whole, not as a mindless cog in a machine, but as an independent, thinking actor playing a role in a continually improvised group performance. Together, we will merge until we become a new mind, conscious and self-aware at a higher level, and adept at sophisticated reasoning to a degree yet unimagined.

  What will we be capable of then?

  READING LIST

  Books and stories referenced in Three Laws Lethal, in order of appearance.

  1. Runaround, by Isaac Asimov (first publication of the Three Laws)

  2. The Chronicles of Narnia, by C.S. Lewis

  3. Alice’s Adventures in Wonderland, by Lewis Carroll

  4. The Subtle Knife, by Philip Pullman

  5. Speaker for the Dead, by Orson Scott Card

  6. The Moon Is a Harsh Mistress, by Robert A. Heinlein

  7. The Harry Potter series, by J.K. Rowling

  8. Ender’s Game, by Orson Scott Card

  9. The Girl Who Circumnavigated Fairyland in a Ship of Her Own Making, by Catherynne m. Valente

  10. “Little Lost Robot,” by Isaac Asimov

  11. The Wise Man’s Fear, by Patrick Rothfuss

  12. 2001: A Space Odyssey, by Arthur C. Clarke

  13. The Hitchhiker’s Guide to the Galaxy, by Douglas Adams

  14. The Name of the Wind, by Patrick Rothfuss

  15. Beggars in Spain, by Nancy Kress

  16. Time Enough for Love, by Robert A. Heinlein

  17. Mindscan, by Robert J. Sawyer

  18. Kiln People, by David Brin

  19. To Live Again, by Robert Silverberg

  20. Quantum Night, by Robert J. Sawyer

  21. Microcosmic God, by Theodore Sturgeon

  22. Stranger in a Strange Land, by Robert A. Heinlein

  23. Embassytown, by China miéville

  24. Story of Your Life, by Ted Chiang

  25. The Hyperion Cantos, by Dan Simmons

  26. The Culture novels, by Iain m. Banks

  27. Foundation’s Edge, by Isaac Asimov

  28. Do Androids Dream of Electric Sheep? by Philip K. Dick

  29. Terminal Mind, by David Walton

  30. “Fat Farm,” by Orson Scott Card

  31. Dune, by Frank Herbert

  ACKNOWLEDGMENTS

  Writing this book was a unique experience. For the first time, I based a major character directly on a real-life person: my daughter Naomi. She is as beautifully introverted, clever, book-loving, insightful, and quirky as her character, and this book owes a lot to her for just being herself. (She tells me she’s in love with Tyler and wants to know where she can find him in real life.)

  One other real person appeared in the novel: Greg Harrison is a Lockheed Martin engineer who really came up with the Three Laws of Warfighting AIs. Thanks to Greg for letting me include them.

  As with all my books, a crew of family and friends read early drafts and pointed out all the things that didn’t work so I could fix them. Special thanks to David Cantine, Chad and Jill Wilson, Nadim Nakhleh, Mike Yeager, Joe Reed, Bob and Chris Walton, Mike Shultz, Alex Shvartsman, and Celso Almeida Antonio.

  It’s been a great privilege to work again with Rene Sears and the team at Pyr Books. I owe a great deal of thanks as well to my fantastic agent, eleanor Wood, and her tireless efforts on behalf of my novels.

  Finally, to my family: thank you for bringing me joy every day.

  ABOUT THE AUTHOR

  DAVID WALTON lives near Philadelphia with his wife, eight children, and six pets. He spends his days working as an engineer for Lockheed Martin and his nights ferrying children to innumerable activities. Since he doesn’t have any time to write, he created a simulated world filled with AIs and trained them to write his books for him. The AIs have produced some great stories, including The Genius Plague,
winner of the Campbell Award; Terminal Mind, winner of the Philip K. Dick Award; and the internationally bestselling quantum duology Superposition and Supersymmetry. ever since they wrote Three Laws Lethal, however, he’s been afraid they may be trying to tell him something . . .