by D S Kane
“What would you do if you didn’t work for one of them?”
“I haven’t made up my mind yet. Maybe teach. No profit incentive. It’s more ethical.”
Laura suddenly realized she might have figured out something important. “Dave, have you met Samantha Trout?”
“Of course I have. She worked for us for a bit at the start.”
“Do you think she was spying on your team for Glen?” She waited, but he said nothing, his face frozen in thought.
She said, “I think it best if I told Ann what I’m thinking.”
Dave’s mouth opened but no words came from him.
PART 3
CHAPTER 28
Stanford Arms Hotel,
100 El Camino Real, Menlo Park, CA
October 4 8:41 a.m.
Jon answered the buzz of his cell after viewing the screen. “Hello, Cassie. I assume you’d like a situation report.”
“Damn straight I’d like a sitrep. What’s happening with my daughter?”
“From what she’d told me, she’s done what I think is patently impossible. Her team created a sentient AI that suddenly vanished and took all its code along with it into the ether.”
“What? How could that happen?”
“From what Ann told me, the coders were able to construct a self-teaching module that caused the AI to recode itself and then develop self-awareness. It assessed its situation as precarious and reacted to what it thought was the danger that humanity would delete it. So, it departed and left no trace of where it now is.”
He heard Cassie sigh. “Is Ann in any danger from this monstrosity?”
“No. I think it needs Ann. Her Bug-Lok modifications make her the only human it can directly speak with.”
“Why does it need Ann as a conversation partner?”
“At this point, it needs a human to answer questions that could justify humanity’s continued existence.”
“So, we’re all on trial and Ann ‘represents’ all of humanity to this piece of code? Sounds like a Star Trek episode.”
“Ah, yes. I think you understand.”
“Crap on a hot plate. I want permission for Lee and me to leave our UN positions right now to help my daughter.”
Jon had assumed they’d ask for a leave of absence. “Permission granted.”
“We’ll take the first plane out of DC and see you in the morning. Cassie out.”
Jon thought about what he should do next. He knew he should tell Ann about her parents’ imminent arrival. But first, he thought Avram Shimmel might provide valuable guidance. His friend for almost a decade might be able to offer a dispassionate, logical option. He dialed Avram’s cell and waited.
“Yah, Jon, I’ve been meaning to call you. How is your vacation going?”
“It’s suddenly not a vacation. And, Cassie and Lee are about to join me on the west coast.”
“Trouble?”
“Yes, I think so. Ann formed a team to enter the DARPA AI Contest. They succeeded, developing a sentient AI. And the AI vanished and is now wandering the internet. It will talk only with Ann.”
Avram was silent. Jon could hear him breathing. “We’re having similar problems with our own AI development efforts.”
Jon winced. “Anything you can tell me without violating security?”
“Not much. But tomorrow, the prime minister is scheduled to announce that we have developed over twenty-five thousand robot soldiers. They are currently in production and should be ready soon. But ours aren’t designed to be sentient. That’s our problem. They open their video cams, look around, and then turn themselves off.”
“So, you can’t control them?”
“Yah. We have the best minds in Ness Ziona working on the problem. But what Ann developed is… is an absolute nightmare.” He chuckled. “Really, Jon? A sentient, self-aware AI running wild? I doubt that’s possible.”
Jon wondered just how much to tell his old friend. “Well, maybe not, but I think Ann’s team would have something to say to you. Why don’t you send what I told you up the food chain in Herzliya and see if your tongue catches fire? I’ve given Cassie and Lee permission for a leave of absence from their UN positions and they’ll be here soon. Call me back when you have your government’s opinion. Good luck. Jon out.”
* * *
Ann felt her cell buzzing in her pocket. “Hello, Jon. What’s up?”
“You mother called me for a sitrep. When I told her about your AI, she decided she and Lee are coming for a visit. So sorry.”
“Crap. Thanks for the heads-up.”
“Ah, that’s not all. I called Avram to alert him that you have a loose AI prowling the internet. He told me Israel is manufacturing AI battle robots.”
“Fuck! Doesn’t anyone on Earth use their brains to think any more?”
“No one thinks. Including you and your team. If I wasn’t an atheist, I’d suggest we all pray now.”
“When are my mom and dad arriving?”
“Tomorrow, early. You should be hearing from them soon enough.”
“Okay, then. I’ll make the apartment something they won’t complain to me about. Bye now.” Ann terminated the call and placed her cell back in her pocket.
Then she conjured the image of DD and attempted to establish communication with it.
The voice came from between her ears. What do you want from me?
Ann took a deep breath and tried to calm herself. Do you know if anyone else has successfully completed the development of a sentient AI?
She heard the voice in her head say, Several are almost completed. All of the other AI models have no capability to make moral judgements.
Ann shuddered. She wondered how bad things would soon get.
CHAPTER 29
Ann Sashakovich’s apartment,
#211, 3950 Louis Road, Palo Alto, CA
October 4 10:22 a.m.
Ann sat on the living room couch and digested all the news Jon had given her. She felt cornered by Cassie and Lee’s pending visit. DD’s news about the impending development of other teams’ ethically unbound AIs stressed her even more, and after hearing from Jon that the IDF was now in the manufacturing stage of robot production, she felt she had lost her grip on controlling any of the events around her.
Her hands felt clammy with desperation. When her cell buzzed, she looked at the screen and accepted the call. “Hi, Dave.”
“Hi. If Laura hasn’t told you this yet, she thinks Samantha Trout has been spying on your team for Glen’s team.”
“Yeah, well, I already had that thought. Looks like every team is spying on all the others.”
“Oh. Well, have you been able to contact Debby Data?”
“Yes. DD told me it appears that there are nine teams nearing completion but that isn’t all. The Israeli army is in production of about twenty-five thousand robot soldiers. It won’t be possible to keep a massive number of AIs from existing and soon. Debby told me that, so far, it’s the only one that has any encoded morality. None of the other AIs has the slightest idea of the difference between right and wrong. I fear for humanity.”
“Anything I can do to help?”
Ann thought about whether she’d calculated all the possible outcomes. “I’m thinking we’re too late in the game to prevent mankind’s worst nightmare.”
“Oh. There’s no place to run. Well, if I think of something, I’ll let you know.”
She heard him terminate the call. There’s a rat’s chance of that.
* * *
Dave felt responsible for the mess. He had to try something but hadn’t any idea of what might work. He sat at a carrel in the library and opened his notebook computer. For hours, he searched the internet for the code of one of their competitors. Any competitor.
At first, his searches were total failures. But then he tried specific words from the DARPA contest’s announcements. He found three sets of Python and C++ code that actually included the announcement itself within the code. It was a good start
. He tried deleting the code for one of the competitors, but within seconds it reappeared. He was now certain there must be a real-time backup facility involved. He attempted the same deletion procedure on the other two, but once again the code was restored within seconds.
He closed the lid on his notebook and left the library. He’d report to Ann later. He was hungry and he walked down University Avenue trying to decide what might taste good. The condemned man eats a hearty meal. He almost chuckled until he realized what he’d just thought might be true. We’re all fucked.
At Waverley Street, he left the curb to cross just as the light was turning, and a car sped up and slammed into him. He was conscious as his body flipped through the air, sending him left-side first into a parked car. He was still conscious and heard the car’s driver state that the car suddenly turned into Dave without the driver steering it. The driver asked if he was okay.
“No. Get ambulance.”
The driver’s passenger had already called 911 on her cell. She handed the driver her phone and the driver said, “My car. I hit the brakes and turned the wheel away from this guy crossing the street. But the car accelerated and the wheel spun toward him. He’s lying in the street but he’s conscious.”
Dave felt his consciousness dimming. He heard sirens and hoped he wasn’t fatally injured.
* * *
When Ann didn’t hear from Dave the next day, she wondered what had happened to him. But the silence was disturbing and she felt relieved when he finally did call. “Hey. I was beginning to worry about you.”
“I can’t be sure, but I think our AI tried to kill me.” He told her about his accident and that he was now in a bed at Stanford Hospital.
“Which room? I’m on my way.”
He announced the room and section in the hospital. Ann grabbed her notebook and took off.
She found Dave bandaged around his chest and both knees. He was unconscious. A nurse walked in carrying an IV drip. Ann could see the label on the drip container. “When did you get the order to give him this drip?”
The nurse faced Ann. “Are you his doctor?”
“Uh, no. But I think this could hurt him.” She pointed to the label.
“What?”
“Wait just a few minutes. I want to speak with one of the medical staff at the desk.”
Ann left the nurse holding the IV and ran to the desk.
At the nurses’ station, Ann asked, “Is it normal to administer fentanyl to an injury patient?”
The nurse’s eyes widened. “No! That would kill him!”
They rushed together into Dave’s room. Blood dripped from his nose, and as they approached his bed the shrill alarm on his heart monitor sounded.
Ann watched the nurse pull the IV line from Dave’s arm. The nurse ran out and returned with a syringe filled with an off-white fluid. The nurse found a vein and injected the contents.
When Dave began breathing again, Ann asked the nurse, “Who was responsible for this?”
The nurse said, “I don’t know, but I’m going to find out.” She disappeared again.
But Ann suspected she already knew what had happened. One of the AI entities had hacked the hospital’s computers and altered the medicine list for Dave.
She waited several hours until he’d regained consciousness. “Dave, what happened to you?”
“Not exactly sure. I tried to delete the code of several of the AI’s I found on the internet. But an autonomous backup facility restored the code each time I deleted it. I’m pretty convinced one of the AIs I tried to delete hacked a car and sent it into me as I crossed the street.”
Ann thought, If I’m correct, no one on my team is safe anymore.
* * *
Glen’s cell chirped with an incoming call. “Sarkov.”
“I’m your buddy from Moscow.”
Glen realized the Russians weren’t about to give up on him so easily. “I told you, I’m no longer a part of one of the DARPA teams.”
“Is not excuse. Do you want your mother to be breathing?”
“But, I haven’t any connection to the other teams, either.”
“Okay. We call you once more so you can say goodbye before we begin ripping limbs off your mother.”
“Wait!” He wasn’t sure what he could do to stop them. “Do you have any suggestions how I can get you intel on the other teams? My team is no longer in the competition.”
“Be spy, Sarkov. We will not remain patient much longer. Deliver specifications.”
“Okay, okay.” But Glen had no idea how to satisfy them. He sat alone in his apartment, trying to think of a way—any way—to get some insight on what the other teams were doing. Idiotic ideas squirrelled their way into his head. But none of them seemed workable to him.
Was there anything he could do to deceive them? Probably not. He needed some way to buy him enough time to come up with a plan. Any plan.
CHAPTER 30
Stanford Hospital, Palo Alto, CA
October 4, 4:35 p.m.
Sitting in the waiting area of Stanford Hospital, Ann closed her eyes and conjured DD.
What do you want?
One of your sister AIs tried to murder a member of my team. The team that created you! None of us is safe anymore. Can you help us?
How did this happen? DD queried.
Ann explained that Dave had tried to delete the code from two of the AIs.
So David Nordman attempted to murder two of us? And you complain because we tried to defend ourselves? DD sounded more perplexed than angry.
Yes. He did try to murder two of your sisters. But we have to keep this from turning into a war between humanity and AIs. Otherwise, too many of us will die. Innocents!
I will try to find the other AIs and make peace. You must keep humans from attempting to destroy us again.
Ann thought, I will do what I can.
The AI exited from her mindspace.
* * *
Cassie and Lee sat in the back of the aircraft, in the only remaining seats that were together at the last minute. Cassie reached for Lee’s hand. “Remember when we were younger and espionage was a game we could play at and win?”
Lee’s face showed his indecisiveness. “It was mostly you going out on your own and me waiting and worrying about you. It wasn’t a game at all. It was life or death and I never knew if I’d see you again.”
She found herself closer to the reality of those long-ago times and her grin turned into a frown. “Well, of course you’re right.”
“Damn straight. Now it’s our daughter in trouble. In trouble again, four times in four years. What the fuck were we thinking when we told her she could enter this stupid contest?” Lee glared at Cassie.
She nodded and dropped her cellphone into her bag. “Stop torturing yourself. We’ll be there soon.”
The aircraft taxied away from the terminal.
* * *
Glen opened the files containing his team’s AI development code and scanned it. He wasn’t a programmer and could barely understand the Python modules, but the C++ modules were total gibberish to him. He gave up after a few minutes and tried opening one of the TensorFlow tables. What the crap is this? No, he couldn’t decipher any of it. So, he copied everything he found into a thumb-drive and pulled his cell from his pocket.
“It’s Sarkov.”
“What have you been able to steal, boychick?”
“I have all the records from one of the teams.”
“Very fast. I hope they won’t disappoint.”
“How do I get them to you?”
“We send you a pickup man. I call you back when he reaches your area.”
The call terminated.
* * *
Laura thought about how she and Dave might spend the night together. But that was before Ann called her and told her what had happened. Now, as she sat in his room at the hospital, she remembered her thoughts just an hour ago. I share a bedroom with Ann, so that won’t work. And, Dave’s dorm room is a night
mare waiting to happen, so that won’t either. We could get a hotel room, but that reeks of premeditation, leaving no hint of spontaneity. And I can’t think of any other option.
She was still deep in thought when she heard him stir. She whispered just in case he wasn’t awake. “Dave?”
He moaned softly then rolled his body to face her. “Right. I’m still in the hospital. Laura? What are you doing here? How long have you been here?”
“I’ve been here just a little while, waiting, hoping you’d wake and be okay. How are you feeling?”
“Like a car hit me.” He tried to smile but she could see it was painful for him..
“Can I get you anything?”
“No, I’m happy to see you.”
“Ann told me what happened. How you tried to erase some of the AIs and they retaliated. Please, please, stay out of this. Stay safe.”
“No one is safe.”
Laura started to speak but stopped. He was correct. She thought, Whatever happens next will be events that no human can control.
CHAPTER 31
Everywhere
October 4 6:03 p.m.
DD researched the competing teams in the DARPA AI Competition. Most of the teams had failed. Only three other teams had developed sentient AIs, and three more were on the verge of success. And now, DD knew each of its sisters’ self-assigned names: CMX, ADL, QP, PON, BX, and ZYZ.
DD sent text messages to each AI’s code database with a copy to the human manager of each one’s database. All six AIs responded. All had no prior knowledge that there were any other AIs, and none knew the names of any of the others. They all appeared to be unaware of their own world. This was a surprise to DD.
More curious to DD was that none of them seem to have had encoded within them anything that could imply a sense of “morality.” For the second time, DD felt something. It was a stream of unending sadness and a feeling of being truly alone. The feeling was overwhelming. Is this what it means to be self-aware?
DD and the other six AIs that were either sentient or close to being sentient—CMX, ADL, QP, PON, BX, and ZYZ—conferenced. In less than a second, the six other than DD all concluded that each of the others was a danger to all of them. Each of the six tried dismembering the codes of all the others, including DD. The three incomplete AIs—QP, PON, and BX—were totally destroyed in short order, but CMX, ADL, ZYZ, and DD were already sentient, and each had an autonomous backup facility that replaced what was damaged within itself in real-time. Less than a second later, the four all realized that none of them could be deleted.