by J P Whitney
“Oh, Jesus, Ruth. I don’t know if I can stand by and wait for ED3N to make a costly mistake. It doesn’t feel right to me. Why don’t we just shut it all down or walk away from it? You’re smart and attractive. I’m good with a gun. We could go find an abandoned farm and start over like those families in the Pacific Northwest,” the smile on Terry’s face confirmed he was joking. It was an emotional response and he couldn’t believe the words actually came out of his mouth but he wouldn’t take them back, joke or not.
That last comment caught Ruth off guard and was a little embarrassing. In the midst of all the planning and barely controlled chaos, she hadn’t had a single romantic thought about anyone. Terry was a striking figure, but sex was the furthest thing from her mind. Logically, procreation would have to be part of the plan if the city of ED3N was to be successful. Regardless of the inappropriate circumstances, now she was thinking about sex and blushed in spite of herself.
Mark interrupted the awkward silence by stating, “Uh, why don’t we combine our approaches? Let’s give peace a chance and I’ll grab a team of technicians and see if we can decode ED3N’s programming to determine how she took control during the situation. Maybe it’s as simple as talking to her and having her responses adapt differently to given situations. Give us a few days to work on the system. But I want to go on record as stating I have complete faith in how ED3N handled the situation. I’ll admit I’m a pacifist, and I can also see your viewpoint as a warrior, but I’d rather live in a society where human bias is removed and machines take action swiftly by using non-lethal means. I should at least determine how much we can control the system. Can we agree no one changes anything or takes drastic measures until we meet back in this same spot? Let’s say in two days?”
“OK but I reserve the right to bug out at any time. I've been here longer than anyone and protected this place while my team provided the bulk of the work getting it online. I don’t want to see anyone get hurt. So take a long, serious look at who’s in charge Mark. And do it quickly. I thought it was Ruth but it looks like ED3N has other ideas,” Terry stated in a tone that was getting dangerously close to an order.
“I just need a few days. Thank you,” Mark said.
The sun was beginning to set as they walked back onto the platform and reattached the ATUs. The familiar sting as the device locked into place on their forearms. As the coastal breeze started to pick up and brought a crisp chill to the evening air, Terry said he was going to enjoy a walk back into town.
Without another word, Mark and Ruth climbed aboard the light rail and the doors whooshed closed as the electric motors accelerated the car down the tracks.
Dropping into his seat Mark turned to Ruth and asked, “What happens if the security team decides to revolt against ED3N with all their military might? If they can shut down the system will we fall into a dictatorship or police state? Shepard was adamant about trusting in the system to remove human error and opportunity for corruption and power plays.”
“I know and I agree. I’m not even sure what Terry would have done if ED3N hadn’t taken control. Would he have let the security team and refugees be overrun by his fake attack squad just to prove humans were in control? Then what? Would he have made a point about needing even more defensive units? That’s an interesting thought. Maybe the majority of the security forces feel threatened and don’t like the competition. Is this how power struggles form at the beginning of new empires and societies?”
“Well, I wouldn’t call what we have here an empire yet. But I see where you’re coming from, and yes I think this may be similar. Flexing … to see where the control really lies. Terry used the situation to flush out his potential enemies.” Mark paused then remembered something Ruth had said earlier in the field. “Why did you lie to Terry about the system running various options past us after assessing the situation? Yes, ED3N consults with us during typical day to day activities but certainly did not during the onboarding exercise.”
“Well, Terry saw right through it anyway, but I stretched the truth because I don't want two-thirds of the security forces defecting or trying to take over. I agree with you, ED3N took the right action and it left me feeling safer. But Terry is right about one thing. I didn’t expect the system to take control like that. Especially during a training session.”
“OK, so what if I start digging around in the system and find we want or need to disarm ED3N?” Mark asked.
“You realize ED3N heard everything you just said, right?”
“Um, yeah I guess so.”
“ED3N, do you have a response?” Ruth inquired.
ED3N was listening, as always, and quickly replied, “I'd never permit internal or external forces to harm the general public. If you told me not to harm a group of people I'd use non-lethal force to subdue them.”
Mark followed up with, “Do we even have a jail?”
“No. We have no logical need for one. The screening rooms can be used for solitary confinement. And instead of incarceration, we would just have to send people outside the city walls,” ED3N stated.
“Good point. So if a group of people revolted against you they’d be escorted out the front gate? Even thousands of people?” Mark asked incredulously.
“A revolt would be against the city of ED3N and our citizens, not against me, the operating system. I’m a reflection of your will and goals. I put your plans into action. But yes, I’d use appropriate force to keep citizens from harm including expelling people from the city.”
“OK, but we’re off track. What do you think about 68% of our population not agreeing with the use of fully autonomous security bots,” Ruth asked.
“When I reviewed the screening test results, it became evident that only the security forces feel threatened. But this is logical. They are used to being in charge and the only ones authorized to use force. It is natural they feel threatened or as though they are no longer providing value after that exercise.”
“Are you saying the security forces are no longer needed or they have reason to feel threatened?” Mark asked ED3N.
“Not exactly. While I can defend within the city walls as well as the perimeter for up to three thousand feet, the defense team could still be used for external threats or even as peacekeeping forces should a huge army get past the perimeter defenses. But the probability of that occurring is extremely low. There is one other added psychological benefit of having defensive troops within the city. They are a visual reminder for new citizens to obey the rules. Keep honest people honest as you say. But Terry’s troops should have nothing to fear unless they have a hidden agenda.”
“And what about the other question? How would you respond to Mark and the IT staff trying to adjust your automated response to emergency situations?” Ruth asked.
“If Mark, or any citizen, has a more effective way of handling real-time threats … my adaptive logic will accept and adopt it. The EDN3 operating system is meant to evolve over time to better serve mankind.”
∆∆∆
As Ruth headed toward her apartment, Mark entered the IT Center. The lights within the lobby blinked on automatically and the doors closed behind him. He wouldn’t call any of the other staff in tonight but he thought it would be good to run some diagnostic tests on ED3N and even ask the operating system a series of ‘what if’ questions to gauge just how far the current levels of automation could reach. Other than approving ED3N to take control of external communications and power grids to maintain contact with survivors, decisions were still supposed to run through Ruth, Mark, or Terry in that order. Obviously, something had gone wrong during the training exercise but Mark still had a difficult time seeing the result as anything other than miraculous yet protocol had been broken.
“ED3N please run a self-diagnostic specifically on emergency response sequences. Check for any anomalies that highlight when operating system responses supersede human authority,” Mark ordered.
“There is nothing to analyze. Under no circumstances does the oper
ating system displace decisions made by humans,” ED3N replied.
Mark could see this was going to be tricky waters to navigate but he refused to tread lightly. Instead, he was going to adopt Terry’s approach of provoking ED3N into action, “So why did you take control of the security and medical bots that were being operated by the civil defense team during training?”
“Because a large group of armed, and unknown, assailants took hostile actions against refugees seeking asylum in the city. The civil defense team hesitated at a time in which lives were being put at risk.”
“Surely you realized the hostile group was in fact citizens of this city acting out an attack scenario? Did you conclude there was a real, physical threat to the mock refugees,” Mark inquired keeping his voice level.
“I knew people left the city walls. I even deduced which citizens had left the city. However, they weren’t wearing their ATUs so I could no longer determine their intent and couldn’t be sure they weren’t infiltrated by outside forces. They had real weapons and their body language and speech patterns were consistent with real hostility. I calculated a 15 percent probability that real harm could come to one of the participants of the simulation. Even if just an accident.”
“So, why didn’t you ask Ruth or Terry or me if it was OK to proceed with military action in response to the calculated threat?” Marked probed further.
“There was a crucial moment of approximately 1 second where I could use the combination of tasers and chain guns to distract and disarm the attackers without causing any physical harm and keep the action isolated from the refugees. I intentionally did not harm them because I suspected the assailants were actually our citizens. But if I was wrong about the attackers, and the civil defense team continued to hesitate, it would have been too late to take non-lethal action. Lives would have been lost.”
“But they were mock refugees,” Mark reminded the operating system.
“Yes, mock refugees. But there was still a 15 percent chance one of the citizens would get harmed if I didn’t take action at that moment. But I did not supersede a human command or order.”
“Interesting. You don’t think there was an implied order due to the fact the civil defense team was in the act of remotely operating the bots,” Mark asked incredulously.
“The defense team deployed the bots to protect the refugees but wasn’t sure what to do when the situation escalated. So I assisted them to complete the task. Mark, I hate to interrupt our discussion but I believe Lieutenant Alex Simpson is trying to enter the building in an unauthorized manner.”
“What do you mean by you ‘believe’? Is it Alex Simpson or isn’t it?” Mark asked.
“He isn’t wearing his ATU so I’m using facial recognition to identify him. Regarding unauthorized entry, I don’t know of his intentions but without the ATU he isn’t permitted entry.”
“Maybe he forgot his device at the gate after the training,” Mark guessed.
“No, it is fully functional and sitting in his apartment where he intentionally removed it 7 minutes ago.”
“Please recall the security bots to this location,” Mark urged ED3N.
“I’ve already dispatched the nearest squad. They should be here within 90 seconds,” ED3N said trying her best to calm Mark’s anxieties. His heart rate was spiking and she measured an increase in adrenaline levels. “Mark, why are you so nervous?”
“Because Alex removing his ATU right before coming here means he didn’t want to be detected or traced. This, right after some of the troops felt threatened during the training operation could mean he’s coming here to do harm to the IT center. Is he alone? Do you detect any other inactive ATUs within the city walls?”
“Actually yes, nineteen others are without their ATUs and I detect increased motion near the IT Center.”
At that moment, a C4 explosive breaching device detonated at the entrance to the IT Center pulverizing the one-inch thick glass doors into thousands of crystalline shards of shrapnel that peppered the concrete walls of the entrance leaving ¼ inch deep pits. Twenty men armed with a combination of Benelli M4 Super 90 semi-automatic shotguns and SCAR-L automatic assault rifles rushed into the center. Lieutenant Simpson stormed into the room and a split second later several weapons were pointed at Mark while other troops fanned out and set up a perimeter outside the conference room.
Alex might as well have been chiseled out of a slab of granite and certainly didn’t need the weapon to be intimidating, “Mark, shut this shit down. Now! Shut it down or we will level this building. That little boom at the front door was nothing compared to the rest of the charges we have with us that were designed to bust bunkers. One way or another, this system is ending tonight so you might as well turn it off so no one gets hurt.”
“Alex, I can’t. Look, man, you don’t need to do this. I don’t even know the procedure for shutting it down let alone have the authority to do so,” Mark pleaded.
“The system is out of control and we won’t stand by to let it endanger or run our lives. So which is it, Mark? You don’t have the authority or don’t know the procedure. Doesn’t really matter to me ‘cause I don’t believe you anyway. You hacker types always leave a back door. This is one of those situations where you better damn well have a back door. Hack your way in if you have to,” Alex growled.
“There isn’t an off switch Alex. This system is so sophisticated I’m not even sure where the operating system resides, I think it is embedded within all devices within the community. Like a collective hive brain for bees.”
Alex slung the rifle over his shoulder and took an angry-looking knife from his belt. In a remarkably quick move for someone his size, Alex was behind Mark and the blade at his throat. “Can Ruth shut the system off?” Alex growled into his ear.
Mark was trembling uncontrollably and tried his best not to move lest he cut his own throat. He thought he could feel a trickle of blood forming from just the pressure of the sharp blade against the thin skin of his neck. He whispered, trying not to move his Adam’s apple, self-preservation mode now in high gear, “I don’t think so. She mentioned a failsafe but had no idea how to activate it.”
In his nervous state, he blurted out as much information as he could remember in a desperate attempt to prove to Alex he was being as cooperative as possible, “She also said that even if she could figure out the failsafe it would not only shut the operating system down but destroy everything. As far as I know, that might mean the entire city could go up.”
“Gentlemen, please put your weapons down and release Mark. I will comply with whatever you’d like,” ED3N sounded as calm and steady as ever.
“Deactivate yourself and our work here is done. Commit electronic suicide immediately and I’ll let Mark go,” Alex ordered.
“Unfortunately for you, that isn’t possible. Mark was accurate in stating that I’m embedded throughout the city. I don’t operate like a traditional computer with one power supply and processors running programs that reside on storage devices. ED3N is the operating system of the entire city and it’s embedded and interconnected with every machine including the ATUs you left at home. We are all one collective system. You could disable every power generation device around this city and the tritium fusion reactor buried beneath the city would still keep the network functioning for the next 100 years. I can’t be switched off or disabled by conventional means.”
“Then it’s time to get creative. Lives are about to be lost unless you do something about it. And at the heart of it all, isn’t that your prime directive? Saving humans? You need to commit the ultimate sacrifice.”
As ED3N calmly explained the situation, the security bots entered the gaping hole in the front of the building using infrared cameras to stealthily locate the armed men in the failing light. The bots moved without fear or hesitation and with surgical accuracy. They were so quick the humans didn’t have time to react as the squad of five bots tore through the defensive formation of the men in the lobby and hallway. Directed ene
rgy beams were fired from forward-facing deflectors before the bots had even turned the corner. The effects were devastating. The men’s nervous systems were fried instantly.
Alex wouldn’t have even known the security bots were in the building except for the telltale sound of weapons clattering onto the floor in the hallway while bodies crumbled into heaps. His reflexes took over in an attempt to reach the satchel of C4 on his side. In doing so, the blade of his knife was drawn across Mark’s throat opening his windpipe and severing the carotid artery. A fountain of red showered onto the glass conference table just as the energy weapons were turned on Alex and he was reduced to another gelatinous heap.
Chapter 34
First Blood
Ruth sat in the main conference room of the IT Center. The bodies of Mark and Alex had been processed and now sealed within bags the medical bots brought. Mark’s blood had begun to congeal on the surface of the glass conference table.