Dangerous Brains

by Erik Hamre

  “So now you have the basic rundown of what we’re looking at,” Ronald Kraut said, rounding off his presentation.

  “The possible extinction of the human race? That’s what we’re looking at?” Amanda asked. She didn’t seem too impressed, she didn’t seem like she was buying it.

  “If this is the case I would prefer to be with my family,” the psychologist, Mike Hanna, said.

  “Extinction is the worst case scenario. We can still stop this,” Kraut replied.

  “How? If what you say is true, then how on Earth are we going to be able to stop this thing? I don’t want my family to be stuck in Hawaii, alone, if this thing escalates.”

  Ronald Kraut stared at the psychologist, unimpressed. “I’ll arrange for an evacuation of your family to a holding centre here in Vegas. They can be held there until the situation is resolved.”

  “What if it isn’t resolved?”

  “It will be resolved. One way or another. But at least you’ll have your family close. That’s the best I can do, Mike. Even if you don’t want to be on the Cronus Team, you can’t leave this place. All of you are now in quarantine. There will be no communication with the outside. Everything passes through me.”

  Mike Hanna nodded. He understood there was no point in arguing. At least he would have his family in the same state if things went bad.

  “Have you tried to communicate with it?” Amanda asked.

  Ronald Kraut shook his head. “Not yet. We wanted to get a better understanding of what we’re dealing with before we attempted that.”

  “And it hasn’t tried to communicate with us?”

  “No, and even if it does we can’t trust anything it says. Just think of the AI Box problem.”

  “The AI Box problem?” Mike, the psychologist asked.

  Kraut sighed. “Why don’t you explain,” he said to Vladimir, before leaving the room.

  Vladimir stood up and walked over to the whiteboard. “The AI box was an informal experiment, conducted by a guy named Eliezer Yudkowsky,” he said, drawing a square box on the whiteboard. “I’m not sure how authentic this experiment really is, as Yudkowsky has refused to release certain details about the experiment itself. But essentially the conclusion is that we can never trust anything smarter than ourselves. Yudkowsky set up an experiment where a hypothetical Artificial General Intelligence and a real human were only allowed to communicate through text-based messages. The artificial intelligence is locked inside a box and its sole goal is to convince the human to set it free, to let it roam loose on the internet. Now we all know the danger of that, so the human should have every incentive to never let the AI out of the box. In the absence of an artificial intelligence, with human level intelligence, Yudkowsky substituted the AGI with a person. The remarkable result of the experiment was that the ‘AI’ managed to trick the gatekeeper into setting it free several times, solely through its text-based argumentation. And when you think about it, it is really not that surprising. Imagine what an Artificial Super Intelligence could promise you; technical innovations that could cure cancer, the end to world hunger, even immortality. It would almost be like negotiating with God. And who wouldn’t be thrown off by that sort of power balance? Our artificial intelligence is already out there, so there is no need for it to trick us into releasing it onto the internet. But it may have agendas we don’t understand yet. Things it needs to acquire to be able to achieve an intelligence explosion for example.

  “The reality is that we are used to negotiating with other people, other humans. We have no way of understanding the possible motivations of a super-smart artificial intelligence. If a human, pretending to be an artificial intelligence, is able to convince another human to let him out of the box,” Vladimir said, pointing at the square box he had just drawn. “Then a real Artificial Super Intelligence will probably be able to convince us to do anything it wants. Including setting ourselves up for our own destruction,” Vladimir said, crossing out the box on the whiteboard.

  “Are you saying there is no point negotiating with it?” Mike Hanna asked.

  Vladimir shook his head. “I’m not. But I think we should be glad it hasn’t tried to communicate with us. The moment it does, we have to be very careful. At the moment it is not stuck inside a box. But it also sort of is. It is stuck inside a million small boxes. And none of them have opposable thumbs or legs to run on. It is limited by its confines.”

  “Wouldn’t some of the robots have tools it could use?” Mike Hanna asked.

  Vladimir smiled. “That’s correct. And it’s a worry. So are 3D printers and a range of other things out there. But we shouldn’t underestimate the complexity required to operate a human body. It’s easy to build a computer that can do a million calculations per second. It is very hard to build one that can walk, talk and observe the world around it like a human.”

  “But that’s what it can. If it has human level intelligence then it should be able to do all those things. Easily.”

  “Not necessarily. It would have to build the equipment to do those things. Right now it is still trapped inside boxes. It is a curious little toddler trapped inside boxes. Boxes we have made. Boxes that will restrict what it can do and can’t. To be truly free it will have to create its own tools, its own creations. And then, then it will be truly dangerous.”

  “Nanotechnology,” Amanda said.

  Vladimir nodded. “And robotics. And a range of other stuff.”

  16

  1st of June 2015

  DARPA’s remote Listening Station No 3

  The Nevada Desert

  DAY 1:

  2355 Hours

  Vladimir rested his temple against the steel door to Sarah Kevorkian’s room. He let out a big breath before knocking on the door, twice.

  “Who is it?” Sarah asked.

  “It’s just me,” Vladimir replied.

  “Come in.”

  Vladimir opened the door, and walked inside. Sarah was sitting on her bed. She was wearing a pair of acid-washed jeans and a green Army sweater.

  “Why did you get me out here?” she asked.

  Vladimir walked over to the basic metal desk in the corner of the room. He pulled out the chair and sat down. “Because right now, this is the safest place on Earth.”

  Sarah closed her eyes. “I’ve just been through another marriage breakup, Vladimir. I’ve got kids now.”

  “I know.”

  “Then why?”

  “I wanted to see you. Is that so wrong?”

  “So you take me away from my kids?”

  “Did Andrew ever find out about us?” Vladimir asked.

  Sarah shook her head. “No. He was always too focused on his work to pay attention to my life. And why should he have cared anyway? We divorced.”

  “You know why. He still loved you. He never stopped.”

  “He always loved his job more.”

  “Is that what you think? I think that this whole thing, this whole artificial intelligence that he has created is about you, I think it is about you and Kevin.”

  “Kevin and me? What on Earth makes you say that?”

  “Think about it. It is the tenth anniversary of Kevin’s abduction tomorrow. Andrew started Neuralgo less than a year after the abduction. I know Andrew, I know him better than anyone. And I’m starting to think he didn’t want to cure death at all. Andrew had no wish to become an immortal. Every day alive was hell for Andrew. He wanted to end it.”

  “He told you that?”

  Vladimir nodded. “He told me he had attempted to take his life earlier this year. He just couldn’t bring himself to go through with it. Didn’t have the balls, he said.”

  “And you didn’t think to get him help?”

  Vladimir shrugged his shoulders. “No, I thought he was just having one of those moments. You know how he was, always bored unless something was happening.”

  Sarah rolled her eyes. “I know. I was married to him for fifteen years. I loved him dearly, but it was hard work being ma
rried to him. He never seemed to appreciate life. Always in a rush. Chasing the next big thing.”

  Vladimir locked eyes with Sarah. “What if there was no next big thing? What if he realised that he couldn’t possibly trump having copied the entire human brain?”

  “So instead of celebrating his biggest achievement in life, he goes on and creates a doomsday weapon? An artificial intelligence that could potentially lead to the extinction of the human race? No way. Andrew might have seemed crazy at times. But there isn’t a bad bone in his body. He would never do something like that.”

  “People do it all the time, Sarah. They can’t bear the thought that their loved ones will live on, and instead of just offing themselves, they kill off their entire family in the absurd belief that they’re doing them a favour – that it will spare them the humiliation.”

  “Andrew would never have done something like that.”

  “He had lost his child, Sarah. He had lost Kevin, the most important person in his life.”

  “That happened a long time ago, Vladimir.”

  Vladimir leant in closer, whispering now. “That’s the thing. Andrew never forgot. Two weeks ago he told me he had come up with a way of settling things, a way of finally settling the murder of Kevin.”

  “What are you saying? Had he found out who did it?”

  “I don’t know. He just said that he would settle it soon.”

  “Oh, my God. Do you really think he would be willing to kill off humanity just to get back at Kevin’s murderer?”

  Vladimir shrugged his shoulders. “Who knows what Andrew is capable of? Who knows?”

  “Have you told Kraut about this?”

  Vladimir shook his head. “Not yet. I don’t know if I can trust him yet.”

  “You need to tell him, Vladimir. Actually, he probably already knows. This entire place is probably bugged.”

  Vladimir shook his head. “They can’t bug it. This part of the building is stripped of any electronics. They are paranoid about Andrew’s creation listening in on them.”

  “What are you going to do?”

  “I need to figure out what Andrew was doing those last few days before all this happened. The answer has to be there. It has to be……”

  Vladimir was broken off mid-sentence by Mike Hanna barging through the door. “You have to come quickly, guys. It’s on again.”

  Vladimir managed to catch a glance at his watch, before heading towards the door. The third attack in 12 hours. The time was 11:56 pm. The first attack had been at five to midnight one day ago, the second at five to 4 am.

  Five to the hour.

  A pattern.

  The first one.

  “Come, Sarah” Vladimir said. “We might need your help.”

  17

  1st of June 2015

  DARPA’s remote Listening Station No 3

  The Nevada Desert

  DAY 1:

  2357 Hours

  The room was filled with soldiers and officers in uniforms. Ronald Kraut was standing in front of a large screen at the far end of the room. Amanda was staring at one of the computer screens with wide eyes.

  “Wow, this is incredible,” she said.

  “How long has it been going on for?” Vladimir asked as he arrived with Mike and Sarah.

  “About one minute and thirty seconds,” Kraut answered.

  “So now we know there is some sort of system.”

  “Looks like it. Five to midnight. Five to four. And now five to midnight again. The question is why there was a four-hour gap between the first and second attack, eight hours between the second and third, and now twelve hours between the third and fourth.”

  “Four plus eight is twelve. We should be expecting the next attack in twenty hours,” Mike Hanna, the psychologist answered.

  “I don’t think so,” Vladimir remarked coldly. “This thing knows we are watching it. It wouldn’t make it that obvious.”

  “You talk about it as if it were human. It’s a machine.”

  “A very smart machine,” Vladimir replied. “And it’s based on a human brain. A very clever one.”

  “Why is it attacking all these computers again? If it has already breached them I mean,” the psychologist asked.

  “We think it’s practising,” Kraut replied. “It’s learning.”

  “Practising for what?”

  “It hasn’t breached our nuclear weapon control systems yet. Those, and the power grids, are the only systems it hasn’t attempted to breach.”

  “Then shut down all your firewalls. Make everything accessible. We have to stop it from learning from our responses,” the psychologist said.

  “It’s already too late,” Amanda said.

  “What do you mean?” Vladimir asked.

  Amanda walked over to the computer screen and pointed at the pulsing lights. It’s not breaching the firewalls. It’s already inside. It’s breaking out.”

  “Breaking out? That doesn’t make any sense. Why would it break out of the systems?”

  “It’s communicating.”

  “Communicating by sending a pulse? I thought this was supposed to be an Artificial General Intelligence. Why doesn’t it just write something on our screens?”

  “It’s not communicating with us.” Amanda walked over to the whiteboard, and picked up a marker. “This artificial intelligence is based on a human brain, correct?”

  “Correct,” Kraut replied.

  “Then what we see isn’t an attempt from the artificial intelligence to breach the systems. What we see is it turning itself on. It’s firing all the neurons at once, just like a brain. That’s the pulse. It’s all the neurons firing at once.”

  “But why expose itself like that? What is it trying to achieve?”

  “It’s looking for something. That pulse is a ping. It is sending out a signal across the world. The only reasons your systems light up is because we detect it. It’s not breaching your systems, it’s escaping them. It’s not trying to learn how to breach your systems, it already knows. It’s already there.”

  “So what is it doing?”

  “It is calling out for someone.”

  “Kevorkian?” Vladimir asked.

  “Could be. It could be that he has set up the system with a trigger. Turn the trigger on. And once the ping gets answered it results in an action.”

  “Or if there is no answer it results in an action?”

  “That is actually a more likely scenario. That might explain the regularity of the attacks. It is counting down.”

  “Counting down to what?” Sarah Kevorkian asked.

  “I don’t know. But it’s counting down to something.”

  Kraut glanced at the vibrating phone in his hand. He recognised the number immediately. He wandered down the hall and out of earshot of the others before answering.

  “Time of incident?” he asked calmly. He didn’t ask any further questions. He just rang off once the person on the other line had delivered his answer. Then he keyed in another number, and waited for the recipient to pick up his phone.

  “Shut down the nanotech industry immediately,” he said.

  “You’re shutting down the nanotech industry already?” Vladimir asked. Kraut hadn’t noticed Vladimir had followed him down the hallway. “What’s happened? There is no need to shut it down this soon.”

  “There is,” Ronald Kraut answered. “And you need to come with me. You all need to come with me. Right now.”

  DAY 2: A ROUGH LANDING

  “For every grain of sand on Earth there are at least ten thousand stars in the universe. That means that for every grain of sand here on Earth, there is at least one hundred Earth like planets out there. Think about that. If intelligent life only develops on one out of each one hundred Earth like planets, then each grain of sand on the entire Earth, would still represent a planet with possibility for intelligent life.

  In the last two hundred years humans have developed technology that allows us to peer far into deep space to loo
k for intelligent life. Yet we see nothing, we hear nothing. As physicist Enrico Fermi famously asked: Where is everybody?

  Many of those Earth like planets would be billions of years older than our planet. Is doesn’t make any sense that they shouldn’t have developed intelligent life. If we continue our exponential development of technology we will soon be able to conquer space, our galaxy and maybe even the rest of the universe. Why has no other alien civilization done that before us? Why is there still nobody out there?

  Enrico Fermi suggested that there could be a great filter, a filter which each intelligent civilization would eventually encounter. What is that filter? Is it asteroids? Is it pollution? Is it ambition? We don’t know. All we know is that when our civilization finally encounters this Great Filter, it can go two ways. We can either end up extinct, or we can beat the filter, and become the first intelligent life to conquer the universe. I believe we will conquer the universe. And I think artificial intelligence will help us do so.” Ronald Kraut, Speech at the Timothy Urbanov Centre for Extinction Events, Sydney 1997.

  18

  2 MONTHS EARLIER

  3rd of April 2015

  Neuralgo Inc’s HQ

  Downtown Las Vegas

  The entire office broke out in spontaneous applause. Some engineers had put on party hats. Others were pouring expensive French champagne into paper cups. Ten years of hard work had finally come to an end. If you had conducted a survey twelve months earlier, not a single one of the engineers in the room would have dared claim Neuralgo would succeed in completing their mission before the end of the following year. But that’s what they had done. In seven short months they had achieved more than they had during the previous nine years.

  One person had never been in doubt though.