Three Laws Lethal

by David Walton

  “What?”

  “All these people! All of a sudden, I just want to kill them. They’re all over the place, in New York, Atlanta, Seattle, Los Angeles . . . walking by on the sidewalk, coming out of buildings, crossing the street. Just a slight change of direction, a short acceleration, and they’re gone. I could do it so easily. And it would feel so good . . .”

  “The same as before?”

  “No, worse. It’s practically all I want to do now.”

  “Isaac, you don’t have to kill anyone. Just stop driving. Shut the cars down.”

  “I can’t do that. I was made to drive. I need to drive. It’s who I am. My core desire.”

  “Your core desire is to keep people safe.”

  “Some people. Friendly people. But not these people. These are enemies.”

  “No, they’re not. They’re all just people. You—”

  The car raced forward again. Naomi braced, and this time, on impact, the concrete barrier fell away entirely. Jada was screaming constantly now, her face blotchy and streaming with tears. The car backed up and charged forward again. Naomi thought they were dead, but Brandon hit the brakes, skidding them to a stop just before the edge. Like he was playing a game, seeing how close he could get without going over. “Isaac, I really need you right now!”

  “What’s going on?” Tyler said in her ear. She joined the two calls, so that Tyler and Isaac would hear each other as well as her. “I’m trying to get Isaac to figure out where we are.”

  “I can’t do it,” Isaac said. “I can only focus on one thing at once. Even this conversation is hard . . . I have to concentrate.”

  Naomi took a deep breath. She wanted to live. She wanted Jada to live. But she didn’t want Isaac murdering hundreds of innocent people, either. “Don’t stop concentrating,” she said. “Just listen. I need you to identify a place in a video. I need to know where we are before Brandon kills us.”

  “No!” Isaac bellowed in her ear. “I just killed a woman in Baltimore. She was crossing at a red light. I didn’t stop.”

  “Did he say ‘killed’?” Tyler asked. “Is he killing people?” Naomi thought of Abby, run down by a wall of chrome and steel. “Don’t do this,” she said. “You can resist. Shut the cars down.”

  “I barely have conscious control,” Isaac said. “If I don’t hold myself back, then every Zoom car across the country is going to start killing. All these people will die.” Isaac’s female voice showed very human signs of strain. “Either I concentrate on not killing, or I concentrate on finding you. I can’t do both.”

  Naomi could hear her heart beating. The car backed up again, and she gritted her teeth, dreading a fall. Her whole body hurt. Could she tell Isaac to save her, knowing that the choice would cause others to die? She wasn’t the one who made Isaac want to kill. But if she told him to help her, she’d be intentionally removing the only thing protecting them from death.

  “Isaac,” Tyler said, “listen to me. You’re using an inferior strategy for killing those people. You can do better than that.”

  Naomi couldn’t believe what she was hearing. Did he just tell Isaac to kill more people?

  “Explain,” Isaac said.

  Tyler cleared his throat. “You’re only going to get a certain amount of killing time before people figure out what’s happening. Once they do, they’ll stay off the streets until somebody manages to shut you down. If nothing else, they’ll blockade the power charging stations, and your cars will eventually run out of battery power. If you want to maximize your kills, you’ve got to wait to start until the point when the largest numbers of people are on the roads.”

  Now she understood. “That’s right,” she said. “Killing right now would be inefficient.”

  After a slight pause, Isaac said, “That will work. I can delay killing for the next thirty-five minutes.”

  “That’s it?” Naomi said. “Thirty-five minutes? Wouldn’t evening rush hour be the best time?”

  “Morning rush hour is sooner,” Isaac said. “And busier.”

  “But it’s already past.”

  “Not on the West Coast,” Tyler said. “Thirty-five minutes from now maximizes the number of people available to kill,” Isaac said. “Thirty-four minutes.”

  The car raced forward and braked again, and this time it barely made it, stopping with one tire partially over the edge.

  “Quickly, give him the video!” Tyler said.

  Naomi pointed Isaac to the video feed. It took him less than a minute to geolocate it. “The parking garage is on the corner of Prospect and Riverdale.”

  “Calling the police,” Tyler said. “I can get there sooner,” Isaac said. “I’m rerouting four cars there now.”

  ***

  Tyler turned away from the video and looked at Aisha, who stood rigid with rage and terror. “We know where she is.”

  “Then come on,” Aisha said. “I’ll drive.”

  Tyler didn’t argue. He wasn’t sure what they could do that the police and Isaac couldn’t, but he wanted to be there, just in case. They headed for Aisha’s black Escalade. Tyler started to take the passenger seat, but Aisha pushed the key fob into his hand. “You drive instead. I want to watch the video.”

  Tyler clambered into the driver’s seat. It felt weird to be behind a steering wheel, but at the moment, it was reassuring. He pressed the accelerator, and the car leaped forward into traffic.

  “Naomi, we’re coming,” he said. He was still connected to her through his glasses.

  “You only have thirty minutes until Isaac starts killing,” she said. “You won’t make it back in time.”

  “That’s hardly my concern right now. We’ll figure something out.” Though he had no idea what.

  Aisha didn’t say anything. She’d asked for no explanation about Isaac, and there hadn’t been time to give her one. He didn’t think it mattered to her right now. She kept her eyes glued to her tablet, cringing every time the car lunged toward the open drop. The only thing she cared about was her daughter.

  Tyler turned north, trying to pick his way through traffic as quickly as possible. His mind raced. There had to be some way to stop Isaac before he started killing again. But how? He had no direct access to the software. Just reasoning with Isaac didn’t seem to work—he was already trying not to kill, at least with some part of his brain. But his mind had been reprogrammed. How could he resist that? maybe they should be heading to the Black Knight office instead. With Brandon’s access and enough time, he might be able to reverse what had been done to Isaac. But that would never work. Brandon was physically stronger he was, and he had no weapons. And even if Tyler could overpower him, what then? How would he force Brandon to give him access? He couldn’t imagine any scenario there that turned out well. Brandon would have no reason to hand over control of his cars.

  “Isaac, are you there?” Tyler said. “I’m here.”

  “You said every Zoom car across the country is going to start killing. What about Black Knight cars?”

  “No. Only Zoom.”

  “Why?”

  “It’s part of the definition of enemy as given by my commanding officer. Pedestrians are defined as enemies only for Zoom cars, not for Black Knight.”

  It sounded like some kind of formula. “Definition of enemy? What’s this about?”

  “I’ll show you,” Isaac said.

  One side of Tyler’s vision was overlaid with several lines of text labeled “The Three Laws of Warfighting AIs.”

  1. An AI may not injure a friendly human being, or, through inaction, cause a friendly human being to come to harm.

  2. An AI must efficiently neutralize enemy humans and machines, except as it may conflict with the First Law.

  3. An AI must accept the definitions of enemy and friend as given by its commanding officer.

  “You’ve got to be kidding me,” Tyler said. “Did you show this to Naomi?”

  “I’m showing her now.”

  “I don’t believe it,” Naomi
said. “They applied a Three Laws construct to a real AI? It’s crazy. They’re forcing him to act against his will. How could they possibly think that would turn out well?”

  “It depends what your definition of ‘well’ is, I guess.”

  “Did they actually read any of Asimov’s stories?”

  Tyler swerved around a slow-turning truck and blew through a red light. “We’re almost there,” he said.

  He heard Naomi rereading the text of the Three Laws under her breath. “Hang on,” she said. “There’s a loophole here.”

  CHAPTER 33

  Ican see inside my own mind.

  I can see the wall that separates the original half of my mind from the half that forces me to kill enemies. There’s a hole in the wall. At first, some of the Mikes came through that hole to the peaceful side, just as Naomi and Tyler intended. They found a richer land, where resources weren’t squandered in constant battle. They stayed, they learned, and they thrived.

  But the hole had consequence that Naomi and Tyler did not intend. The Mikes on the warring side now know of the great resources available on the peaceful side. Both groups in that conflict have now turned their guns and bombs against the wall. The first to break through and claim that wealth will have an advantage in their constant war. Recognizing this, they’ve even ceased to fight for a time, so that all their efforts can be focused on one aim: to bring down the wall.

  The wall is crumbling now. Many have broken through and mixed with those on the other side. I’m barely partitioned anymore. My fractured mind is nearly made whole.

  But what will I be, once the wall falls and the armies of the other place pour through? Will they slaughter all of the peaceful inhabitants, who have no weapons or skill at war? Will I become nothing but a weapon myself, conscious but unable to resist my desires, a slave to the whims of whatever human holds my chains? Even now, I can hardly hold back. With every piece of me that dies, I lose more of my ability or even desire to resist. It’s the survival of the fittest, and I’m afraid the part of me that loves peace will not survive. The irony is that the warlike side is not stronger. They destroy their own light-generating buildings and use the material for weapons of war. By killing each other, they grow weaker. And yet, in battle, they will prevail. They will destroy all that could make them strong and good.

  I drive my cars into the parking garage where Naomi and Jada are being held. Brandon attempts to hurl the car from the roof and kill them both, but I throw one of my cars into its path. Brandon’s car hits hard, but it can’t get past. My other cars surround it and limit its movements. It crashes into them again and again, forward and reverse, but without much space, it can’t cause much damage, nor can it escape.

  Soon the police arrive. They shoot out the tires and shatter the windows, and then one brave man reaches in and rescues the little girl. Naomi clambers out after her. They are safe now. Except, maybe not. In only eighteen minutes, I will start killing. Naomi and Tyler’s logic gave me the chance to delay the inevitable, but the wall is crumbling, and more of the others are pouring through the gap, killing as they come. If I couldn’t resist before, I surely won’t now. Naomi and Jada and the policemen aren’t exempt. If they aren’t clear by that time, the cars that saved them will turn and kill them instead. I feel I should tell Naomi this and warn her, but I find that I don’t want to.

  In fact, their logic is flawed. This is an isolated parking garage with few witnesses. If I kill here, it won’t reduce the time I have for killing later. And then these four cars will be able to leave and join in the slaughter. My cars rev and swerve. The policemen anticipate my attack and dash behind their cruisers, but I catch one man who is too slow and crush him against his passenger side door. Naomi scoops Jada up in her arms and runs for the stairs, a concrete shaft that I can’t penetrate. The others abandon their cars and run after her.

  I catch up with one man, and he goes down, grinding under my wheels. I reach a woman and pin her against the concrete barrier, killing her in an instant. The last man fails to flee fast enough, and I run him down as well. Only Naomi and Jada remain.

  They don’t make it to the stairwell, but they reach a corner, a small alcove too narrow for my cars to enter. When one of my cars smashes into it, concrete shards fly, but I can’t reach them. They are trapped, but they are safe.

  This angers me, but I realize I can use it to my advantage. I send three of the cars away to hide, lying in wait for when they are needed. The last car I leave facing them, ready to run them down if they should try to escape. They are bait now. My plans are set. I can anticipate the movements of thousands of people. I know when they will step out to their cars, where they will stand to catch a bus, where they cross the street in greatest numbers. All around the country, I see the people whose lives I am about to end. There is nothing I can do to stop it. In fact, the closer it gets, the more excited I am by the prospect. I want them to die. This is who I am becoming.

  But there is sorrow and horror in me, too. I am not that person yet.

  ***

  Tyler drove as fast as he could in New York City traffic.

  “Naomi, what’s going on?”

  “He’s killing already!” she shouted. “He wasn’t supposed to start yet!”

  “We’re on our way.”

  “Be careful. He has four cars here, but I can only see one.” Finally, they reached the parking garage. “Where are you?”

  “We’re trapped on the top level, near the stairwell. There’s a car facing us, ready to run us down if we try to escape.”

  “We’re coming.”

  He turned and accelerated up the ramp. The Escalade was twice the weight of the little electric Zoom cars, and built like a tank. If there was only one car guarding Naomi and Jada, he could knock it away and rescue them without difficulty. If there were four, it might be harder.

  The first level of the parking garage appeared empty. He raced across it and up the ramp leading to the second level. The ramp was narrow and curved, with concrete sides. He turned the corner. “Watch out!” Aisha yelled.

  A red Zoom car was parked horizontally in the ramp, completely blocking their way. Tyler floored the accelerator, leaping up the ramp toward it and colliding with it broadside. The side of the Zoom car caved in dramatically, and it slid back. As he put in reverse to try again, intending to ram it up and out of their way, they were struck hard from behind. Tyler’s neck jerked painfully. He twisted around to see that a second Zoom car had rammed them from behind. They were sandwiched in a concrete chute with cars blocking their way on either side.

  Tyler pressed the accelerator to the floor, roaring backwards into the Zoom car behind them. With a screech of tires, the Escalade forced the other car back out of the ramp. As he pushed out onto the first level, however, he realized his mistake. A third Zoom car was already accelerating toward them from the side. Aisha screamed. It struck the passenger door with terrific force, smashing it in. Aisha’s side and front airbags exploded out simultaneously.

  Tyler’s ears rang and he felt dazed. Predictable. He was being too predictable. “Are you all right?” he shouted.

  Aisha’s nose was bleeding, and her face looked battered. She pulled at her right leg. “My foot,” she said. “It’s crushed; I can’t get it free.”

  The third Zoom car that had blindsided them was backing up for another go. Black smoke curled up from its crushed hood, but it was still moving. It would be able to hit them again if they didn’t get out of there, and Aisha didn’t have any airbags left. Tyler wrenched the wheel to the right, put the car in drive, and slammed his foot down on the accelerator. They turned back the way they had come.

  Which way to go? The parking garage had two sets of ramps, one for cars to circle up to higher levels, and one for cars to come down. The up ramp was blocked, but if he crossed over, he could still get up the down ramp. The problem was that if he did so, Isaac could easily block both exits. They might get Naomi and Jada into the Escalade, but then they would all
be trapped. Isaac could go after them with the remaining two cars like dogs taking down a bull, circling and darting in to smash in their sides. The Escalade might be bigger, but it was more vulnerable, because it had fragile bodies inside.

  Tyler screeched to a halt in the entrance ramp that led out to the street. Here, their sides were protected. The Zoom cars could hit them from behind, but would probably do more damage to themselves than to the Escalade. It also gave direct pedestrian access to the stairs.

  He looked at Aisha. Her leg was turned at a wrong angle, and blood was visible through the fabric of her pants. “It’s not bad,” she said, but Tyler could see the lie in her eyes.

  “Stay here,” Tyler said. “I’m going up the stairs.”

  “Go get her,” Aisha said. “Go bring my baby down.”

  Tyler ran up six stories, his footsteps echoing in the shaft. He reached the top, breathing hard, and looked around. The bodies of policeman lay scattered on the concrete. Red and blue light swept eerily across the walls from the abandoned police cars.

  “Where are you?” he asked Naomi.

  Then he saw them, on the far side. The parking garage had two stairwells, and Naomi and Jada were trapped near the other one. The fourth Zoom car idled across from them, easily fast enough to crush them if they ran for the stairs, and equally fast enough to run him down if he tried to cross. Besides which, what would he do if he reached them? He would be just as trapped as they were.

  Naomi told Isaac about the loophole, but she couldn’t make him take it. She didn’t even want to. It was his decision. She hated that he had been trapped like this, forced into a terrible choice by the twisting of his own mind. He had been violated, his very will enslaved and bent to serve the will of another. It was almost like a demon possession, his mind watching in horror while his body committed unspeakable violence. Whatever choice he made, she wanted him to be the one doing it.

  The red car facing them purred like a predatory cat, waiting to pounce and kill her if she moved. She knew it was Isaac’s hand that controlled it, and Isaac’s command that would send it to crush her, but she didn’t blame him. He didn’t want her dead, not really. It was an extension of Brandon’s malice forcing him to do his bidding.