by Erik Hamre
“You still think he did this on purpose. You still think this is some sort of elaborate murder-suicide, where Andrew planned for the world to burn.”
“Don’t you?”
“No, I don’t. I saw him last week. He was genuine when he told me he was looking forward to meeting my girls.”
“You, finally moving on with your life. You, being happy, you replacing Kevin with new kids. Isn’t that exactly something that could have triggered a reaction like this?”
“I don’t think so. There has to be more to it. You knew Andrew. He was a control freak. When we bought our first house in Palo Alto he refused to furnish it for three years, simply because he couldn’t find the appropriate furniture for the place. I lived out of boxes for three years, Vladimir. Andrew was brilliant, but he was also very controlling and dominant.”
“I know. I’m amazed you lasted as long as you did.”
“It was because of Kevin. Kevin kept us together. And when Kevin disappeared we both realised there was nothing left. I didn’t love him anymore.” Sarah wiped a tear away from her cheek.
“So you think Andrew would have left a backdoor, a way to communicate and control the AI?’
“Wouldn’t you?”
Vladimir smiled. He definitely would. Many psychologists believed that one’s personality was a product of environment and genes. Whatever personality Vladimir’s genes had offered hadn’t been much. Vladimir had been a shy and awkward programmer when he first arrived in America. But you became whom you spent time with. And Vladimir had spent most of his waking hours with his mentor, Kevorkian. Although he could never hope to achieve what Kevorkian had achieved in his life, or outsmart him in a game of chess, Vladimir had become a younger version of Kevorkian, an extension of him. In meetings he would often finish Kevorkian’s sentences when Kevorkian got bored or his attention drifted away. Vladimir had acquired an uncanny ability to guess what Kevorkian was thinking. And that was why he was so frustrated. He had never seen this coming. How was it possible that Kevorkian had been able to do all this in his spare time? Vladimir had been impressed with what Kevorkian achieved in the hours he put in at Neuralgo. And those hours often stretched into the early morning. It wasn’t unusual for Kevorkian to put in sixteen-hour days. And now it turned out he had been living a secret life on the side. How in hell had he made time for it all?
Vladimir nodded. “I would. I definitely would.”
“So what could this backdoor be? If you had developed this AI, what measures would you have put in place to ensure that you could always control it?”
“That’s the problem. There are no measures you can put in place. Like you can’t program a child to become what you want him to become, you can’t program an AI to become what you want. Kevorkian pioneered evolutionary programming; self-learning programming. And the problem with evolutionary programming is that it’s not logical. Sometimes the best solutions aren’t obvious, and because of that it is impossible to reverse engineer the solutions. Even if we find the initial program that Kevorkian used to upload Cronus to the internet, it will be useless. It will only reflect an early version of what it is now.”
“So we just have to hope Andrew knew what he was doing?”
“Right now Kevorkian’s creation is an aspiring almighty God in our universe. We just have to hope that it is going to be more merciful than the one we’ve been praying to for all these years.”
Sarah stared at the wall. “I hope he’s changed from when he was married to me.”
22
2nd of June 2015
DARPA’s remote Listening Station No 3
The Nevada Desert
DAY 2:
0500 Hours
Amanda was beating away on the keyboard as Ronald Kraut poured them each a cup of coffee.
“How do we know for certain that the artificial intelligence has surpassed human intelligence levels?” Mike Hanna asked. It was a question that should have been asked earlier.
“There are three distinct levels of artificial intelligence. Artificial Narrow Intelligence is intelligence that specialises in one area. IBM’s Watson computer will most definitely beat you in a game of chess, but it would do a poor job in selecting a birthday present for your wife. Amazon’s intelligent algorithm should help you out there though. Artificial General intelligence on the other hand marks the point when a computer is as smart as you across the board. It can perform any intellectual task just as well as you. That’s where Cronus is at the moment. And we know this because of the detection systems we created two years ago.”
“The Turing Tripwires?”
Kraut nodded. “Cronus only made the mistake of setting off the tripwires once. It has evolved since then. But at least we know that we are dealing with an Artificial General Intelligence, not a cyberattack from a foreign government.”
“So Cronus basically tricked the US Government into believing it was a human being?”
“Nineteen times. All simultaneously at different locations around the world.”
“Shit.”
“That was my initial reaction too. There was no doubt though. At that very moment we knew we were dealing with something different, something we had never seen before. If it had only hacked our computer systems it could theoretically still have been classified as an Artificial Narrow Intelligence, something incredibly skilled at hacking computer systems, and only that. But as it managed to convince our detection systems that it was a human being, we instantly knew it was probably even better than that. And we knew for sure that someone had managed to create at least the equivalent of a human brain. Right now it is only a question of how quickly it will progress to the next level.”
“Which is Artificial Super Intelligence?”
“Correct. An intellect smarter than Einstein and more creative than J.K. Rowling. An intellect that will outsmart and outperform the best human brain in every conceivable field.”
“How much smarter is it now?”
“It doesn’t really matter how much smarter it is now, because unlike humans, who have a natural limitation to how smart we can get, there will be no limit for an artificial intelligence. It can improve itself indefinitely. Humans have to wait for hundreds of years for even miniscule changes to take effect in our neural connections. A computer can just tweak the program, and evolve. We don’t know exactly where the threshold will be. But at some stage an Artificial General Intelligence will reach a level where there will be an intelligence explosion. It will make itself so smart that we don’t even have the mental capacity to understand what it has become.”
“What do you mean?”
“Have you ever seen a fly sitting on a dinner plate?”
Sarah nodded.
“The fly may understand that the crumbs left on the plate are food. But if it were to raise its gaze and look across the floor of the restaurant the sight would mean nothing to it. It wouldn’t be able to appreciate all the small things that go into running a successful restaurant. And if it decided to take to its wings and fly across the room, eventually banging into the window on the other side, the many high-rises and airplanes in the skyline outside the glass would be totally foreign to it. And that’s how an Artificial Super Intelligence eventually will be to us: Incomprehensible.”
“I don’t understand,” Amanda said. She had been working the keyboard for a good ten minutes, and the computer screen was filled with line after line of code. “If this fucking artificial intelligence, this Cronus, is so super smart, why doesn’t it communicate? Why doesn’t it respond when we reach out to it? Surely it’s capable of sending me an email if it can sneak into every fucking computer system in the world.”
“Would you have answered if the FBI had reached out for you, back when you were snooping in their systems?”
“I’m not the fucking FBI.”
“It doesn’t matter. Unless we have something to offer it probably won’t respond or reach out to us.”
“So why are we wasting our time doing
this?”
“We’ve got nothing better to do at the moment. All we can do is to continue to reach out, and then maybe, maybe we can learn something from its behaviour.”
“Well you don’t need a hacker then, do you? You need a bloody psychologist.”
“Perhaps. That’s why we’ve got Mike on the team. But it won’t take long before Cronus will have very little in common with humans, if it ever has had anything in common with us at all. Anyway, a hacker is probably a lot better equipped to understand the responses of a machine than a psychologist is.”
23
2nd of June 2015
Ministry of Technological Warfare
Central Moscow, Russia
DAY 2:
0600 Hours (1700 Hours local time Moscow)
Major Olokoff rushed across the concrete floor of the Russian command centre. “Are you certain?” he asked the person on the other end of the phone line. Olokoff stopped for a second, listening to the other person’s reply, then he abruptly put his phone away.
Major Olokoff had been recruited from the prestigious Moscow Institute of Engineering at age twenty-seven. He was a major by rank, but he had never actually led any troops in combat or even participated in any real military exercises. The rank had come with the job. Career officers were a strange species. Often all they cared about was the rank. Credentials and intelligence didn’t matter. They obeyed the people with a higher rank than themselves. That was it. Everyone else they bossed around. Having a formal rank of Major at least elevated Olokoff above most private contractors and scientists working for the military. But he was hardly at the top of the pecking order.
Most people in the command centre were generals.
Major Olokoff closed his eyes and took a deep breath. When he exhaled his chest hurt. He had been sick with the flu for the last four weeks. The worst was over, but he just didn’t seem to be able to bounce back as he used to. One bout of a bad pneumonia five years ago had made every flu season since hell. There was obviously something wrong with his immune system. Maybe the pneumonia had weakened it, kind of like a malaria attack sometimes did? He’d had malaria as well, in his youth. Whatever the cause, it was starting to annoy Major Olokoff. He didn’t have time to be sick, didn’t have the patience to deal with a body that didn’t function at one hundred percent.
He had just been on the phone to the SVR. They had reported that the sleeper at Neuralgo Inc, Vladimir Sorovis, had disappeared more than twenty-four hours ago. Nobody knew where he was. A coincidence? The SVR didn’t think so. Neither did Major Olokoff. The general had been correct. There was most likely a connection between what had happened at Neuralgo Inc and the first Artificial General Intelligence in recorded history. Now the question was where Vladimir Sorovis’ loyalty lay. He had never been contacted in the twelve years he had been in the US – Neuralgo had never been considered a likely candidate to make a breakthrough in artificial intelligence research. They were a mapping facility. Their project had been to copy every neural connection of a human brain.
Major Olokoff had never considered that research to be a risk.
Machines were different from humans. Computer code was different from biology. If one wanted to make a perfect airplane one didn’t just blindly copy the way a bird flapped its wings. One designed the plane using a machine-oriented approach.
The other issue was that Neuralgo Inc didn’t possess the raw computing capacity to run a human brain. Raymond Kurzweil, Google’s Director of Engineering, had estimated that the human brain’s raw calculating capacity was about ten quadrillion calculations per second. The world’s fastest supercomputer, China’s Tianhe-2, had actually beaten that number, clocking in at about thirty-four quadrillion calculations per second. But Tianhe-2 had cost $390 million to develop and was the size of a very large house. It was unlikely that Neuralgo Inc or any other private company would be in possession of equipment sufficient to run a copy of a human brain. A brain of a mouse maybe, but not a human one. The human brain was a technological miracle. It fit inside our tiny skulls and only required around twenty watts to run, the same as a lightbulb. China’s Tianhe-2 required twenty-four megawatts of power to run, the same power required to run almost 4,000 homes.
Olokoff coughed so hard it felt like he cracked a rib. Was it possible that Neuralgo had succeeded in mapping the entire brain? It couldn’t be. Two years ago they had only mapped around one percent. Olokoff knew these things usually worked exponentially. But surely they wouldn’t have been able to ramp up their mapping efforts on a scale like that? Something else must have happened, if the Artificial General Intelligence originated at Neuralgo.
A paradigm shift.
“Have we got hold of Vladimir Sorovis’ brother yet?” Olokoff asked.
The soldier nodded. “He will be here in twenty minutes.”
“Good. Put him in the interrogation room. I will conduct the interview myself.”
24
2nd of June 2015
DARPA’s remote Listening Station No 3
The Nevada Desert
DAY 2:
0700 Hours
Vladimir was leaning against the wall. He looked tired. “I don’t understand how he made the leap. What we did was to copy his brain. We didn’t attempt to create a self-improving artificial intelligence. We just wanted to emulate Kevorkian’s brain, and then see what we could learn from it. Then maybe in twenty to thirty years we could maybe hope to be able to upload a person’s memories and personality to the cloud, thus curing aging.”
Kraut stretched his back. He could hear it crack as he raised his arms towards the ceiling. “That’s the thing with technology. You never know how close you are to making a breakthrough discovery. Kevorkian may only have made a small tweak or two to the system, and that was all that was required to make Cronus conscious.”
“Conscious? You don’t believe Cronus is conscious, do you?” Vladimir asked.
“Don’t you?”
“Hell no. I don’t think it is possible to ever make a computer conscious. Whatever Kevorkian managed to create, it is still a machine, driven by basic goals.”
“And what are you? You believe you have free will? That nothing you do is hardwired into your genes?”
“There’s an element of hardwiring, but I’m still able to make conscious decisions. Maybe my hardwired goal from evolution is to reproduce, to spread my genes far and wide. But I can choose not to. I am the captain of my soul, as William Ernest Henley so poetically wrote.”
“Do you really think so?”
“Yes, I do.”
“What if consciousness is an emergent property of a complex system? All mammals and birds seem to at least have some level of consciousness. The human brain is much more complex. Thus we automatically assume that we are special. But if consciousness is correlated to the level of complexity of our brains, then there is nothing preventing a machine brain from attaining consciousness. Especially not if it is modelled on a biological brain.”
Vladimir stared at Kraut. Kraut had just stolen Vladimir’s view of how consciousness could develop. He was obviously trying to convince Vladimir of something. Building the argument up so that it made sense.
“This is getting too abstract for me. What is it you’re trying to say?” Vladimir asked, not wanting to play Kraut’s game.
“If consciousness is a property of a complex system, then we need to look at the most complex computer systems we have developed. That’s where we will find Cronus.”
“At NASA?”
“The most complex systems available to private organisations.”
“Wall Street?”
Kraut nodded. “What do you know about Kevorkian’s High Frequency Trading Company?”
“Not much. He used it to look after his money. Kevorkian didn’t trust men in suits.”
“He didn’t trust anyone, by the sounds of it.”
“That’s probably not far from the truth. So what about Kevorkiana HFT? You wouldn’t bring it up unless
there was something about it.”
“Amanda is checking it out as we speak. It turns out the company has spent a lot of money over the last year though.”
“Isn’t that what those companies do? I remember reading that Michael Lewis book, I can’t remember what it was called, but it claimed that almost every HFT company on Wall Street was willing to pay millions to set up operations inside the stock exchanges. Apparently a nanosecond counts when you compete on speed."
“That’s the strange thing. Kevorkiana HFT has never been located on Wall Street. It has never been located near any of the stock exchanges. It turns out it is located in Silicon Valley, just down the road from Kevorkian’s house.”
“Are you kidding me?”
“I’m afraid not.”
“We’ll have to get down there.”
“Transport is being organised as we speak. You wondered were Cronus was launched from. My guess is it was launched from Kevorkiana HFT. My guess is that he has used this company as his cover the entire time.”
25
2nd of June 2015
Kevorkiana HFT’s HQ
Silicon Valley, California
DAY 2:
0800 Hours
The building housing Kevorkiana High Frequency Trading was almost as anonymous as Neuralgo Inc’s head office had been. Just smaller. A lot smaller. A basic two-level structure made out of grey concrete. No windows on the ground floor level. A few tiny ones on the first floor. Parking spots for around twenty cars outside.
“No signage on the door. Isn’t that a bit unusual?” Vladimir asked.