“I have an exceptionally good memory,” he finally continued. “It’s how I managed to keep up with my studies at Cambridge, even though I was spending so much of my time working on Missile Storm. Of course, it turned out that I didn’t even need to graduate … but I’m getting away from myself. The point is I have a good memory. If I didn’t, so many things would have turned out differently.
“Blessing or curse? History will have to be the judge of that. But I saw that code, and nothing was ever the same again.
“I was aware of the pressure of the table on my back, and the smell of antiseptic, and the surgeon’s eyes, peering over the green surgical mask, staring into mine, and that time was slowing down in a woozy, dizzying swirl and I thought that I was just lapsing back into anesthesia, but then my vision seemed to focus in with pin-sharp clarity, and suddenly there was another layer to the world, or I was seeing through the surface of things, gaining a glimpse into some hidden realm. It was like I had a digital overlay, or a VR head-up display, superimposed over my vision. Although that’s got to be a modern revision of the event. I doubt I’d even seen an HUD back then. Of course, I knew that I was hallucinating. I was halfway between life and death, halfway between awake and anesthetized, what other explanation was there?”
“An explanation for what?” Ani asked. “What did you see?”
“I saw three lines of computer code,” Dorian said. “Although, that makes it sound so much less … profound than the experience truly was. I say I saw code, because that’s as close as I can come to explaining it, but even the word saw there makes it seem so much more prosaic than the truth in that moment. It was like I experienced the peeling away of a layer from the onion skins of our reality. For the first and only time in my life, I caught a brief, tantalizing glimpse of the numbers, the math that underpins the universe.
“Perhaps you’ve heard the word palimpsest? It describes a page or papyrus from which the text has been scrubbed, scratched out, or erased, and new text has been written over the top. I saw, in that moment, that what we think of as “reality” is nothing but a palimpsest written over the true text of the universe. I believe that I saw the briefest glimpse of the source code that underpins the universe.
“That’s not what I was thinking then, in that moment. I just saw code, three lines, no more. But as the surgeon looked at me, and he registered that I was awake, and as he decided that he needed to take action, I saw the code change. Watched it transform into three lines of different code. It was like I saw the programming language that hid behind a single human decision. It was mere seconds before the anesthesiologist increased the dose and I went back under, but those lines of code burned themselves into my brain.
“When I woke up in recovery, the memory of the code was there, fresh in my mind. I didn’t even need to ask for a pen and paper.
“And that code seemed so full of possibilities to me. It was mere kilobytes of information, a snippet at best, but the feeling persisted that I had seen the code that ran beyond reality, that I had spotted a conscious decision being made, modeled in the language of computers.
“It was days before I regained the strength and ability to log onto one of my computers. All that time, I was turning those lines of code over and over in my mind, two self-contained routines that connected together as an almost binary pair. Except instead of them being 1 and 0, or off and on, this pair was about something more abstract.
“A decision.
“I remember my hands were actually trembling when I inserted the three lines into a block of my own code. It was a section that controlled the sprite’s move-pattern, governing an enemy’s interactions with the player’s own character’s sprite: Centipeter. The enemy was the AntLion, perhaps you remember it, although perhaps not. You all seem a bit young to have played Centipeter. I ran the program and nothing happened. I don’t know what I’d been expecting. I watched the AntLion for a while, trying to see if its move pattern had become smoother, but I can’t honestly say that it had.
“So, it was all a crazy hallucination, I thought, and just got on with making the game the old-fashioned way.
“We were working on an early build of the game, and I was trying to catch up with the small amount of progress my team had made while I was recovering. I’m pretty sure that I shared the AntLion sequence by accident. I don’t remember doing it consciously. We just kept on refining elements, designing new levels, and working on making the game feel … unique.
“I was out of the office when I got the call that changed my life. Tim Graham, my senior programmer, was playing through what we had, and he rang me up, really excited. He wanted to know about the new algorithm, and how I had come up with such a simple, yet elegant, solution to the “move” problem—the jerkiness of sprites—and how it had also improved the efficiency of the “pursuit” aspects of the enemy sprites’ behavior.
“We’d spent a lot of time trying to make enemies seem intelligent, devising programming strategies that replicated autonomous thought. But they were just algorithms. Instructions. ‘Turn toward enemy,’ ‘hide behind scenery,’ ‘lean out,’ ‘fire,’ ‘hide again.’ Believe it or not, that’s what Dorian Systems was passing off as enemy AI in those days.
“Tim had just got fragged by the AntLion when he called me. AntLion was an end-of-level boss—our first big showy set-piece in the game. She seemed to have clever strategies, but they were only programming tricks. She basically had three different modes that she fell into when the player made particular choices: Chase Mode, where she pursued Centipeter, aiming for a randomly generated number of pixels away from him; Attack Mode, where she let loose with her pincer and slime attacks; and Retreat Mode, which she fell into when she’d taken a certain percentage of damage, letting her regain energy and power up her attacks.
“Because of the simplicity of the programming, she was a pretty easy enemy to beat when you knew her patterns; and we even telegraphed which mode she was entering with facial animation.
“But Tim had been killed by her five times in a row. The strategy he’d always used to defeat her was no longer successful. At the start of the battle, the AntLion had quickly gauged Tim’s strategy—wait, move, fire, wait, move, fire, retreat—and disrupted it by adding a new mode to her arsenal, Shadow, where she mimicked the movements of the player and then, when she was close enough, dropped into attack. It was hardly an earthshaking move, and after five plays, Tim had refined his technique and reclaimed his advantage, but it made me examine the source code.
“Subtle changes had been made to the AntLion’s code. I studied it and couldn’t see how it could account for her adapted movement pattern, the changes almost seemed arbitrary. And the block of code I’d inserted—the code I’d seen in the operating theater—was different, too. It had gained a few new lines, which again, seemed like accidental changes, a corruption of data rather than an improvement to the code. I removed it, saved it, and then played Centipeter. The new move pattern of the AntLion remained. It was, I have to say, a fairly minor improvement, but that it was any improvement at all was worrying. Funny thing about computer code: it’s a lot easier to write it than it is to read it. I knew that the code was different, but I couldn’t see how it could have been responsible for a change of strategy for AntLion.
“I still can’t tell you how it was done. And I am so many generations past that initial implementation of the emet code. Yes, I named it emet. It seemed to animate the AntLion, so the word written on the forehead of the legendary golem seemed entirely appropriate. I didn’t know what I had then, but it didn’t take long for me to learn.”
“You’re telling us that emet improved your code?” Ani asked, incredulously.
Dorian ignored her.
“I pasted emet back into the game’s code in a couple of other places, and watched the improvements that the code wrought. It was slick. Really slick. A few lines of code advanced the game by years. And every time, emet mutated. Grew. Became more complex. Less like the code I’d
input. And every time, after I removed it, the game was better. Faster. Smoother. Cooler. And every time emet grew.
“It was by degrees that I withdrew from the Centipeter project and started working on emet. I delegated more. People thought it was because of the aneurysm, and I let them believe it.
“The truth is, I had become obsessed with this code.
“It evolved.
“It was starting to fit the criteria for an emergent AI.
“People have been talking about constructing an artificial intelligence for centuries. The golem is pretty much a folklore version of one. Automata that sought to emulate human thinking were prized by kings. The idea that computers will become sentient and self-aware is as old as computers themselves. My feeling was that such a computer just got a whole lot closer every time emet adjusted the parameters on Centipeter because it, in effect, mutated, altering itself each time.”
“No wonder it was so hard to beat.” Joe said.
Dorian ignored him, too.
“I built a dedicated computer for emet. I ran game code through that computer—no cut and pasting now—and emet polished it. Improved it. I even started communicating with it—in machine code—instructing it on the areas that needed improving. If my team was having problems with a section of the game, I’d run it through emet, and they’d applaud my solution, even though I had no idea how that solution had come about.
“emet continued to grow. To learn. It outgrew its digital home partway through Granthna’s Revenge. Of course, by then, I was no longer writing code. I didn’t have to. emet was already way better at coding than I could ever become. All I needed to do was suggest parameters, specify size and complexity, get my staff to work on theme and characters, textures and plot. emet did the rest, pretty much by itself.
“emet’s second home was a bigger and better computer system. Moore’s Law: the hardware just keeps getting better. I can’t tell you how many homes emet has had since then. It’s been growing exponentially for decades. Not all of its evolutions have been steps forward. I’ve needed to train it, to guide it, to bring it to life …”
“Paging Doctor Frankenstein,” Ani said, grimly. “How did you decide that teaching emet to become an autonomous AI was a good idea?”
“I don’t expect you to understand,” Dorian said curtly.
“No, I understand,” she replied. “I just wouldn’t have decided to pursue it. Not without someone else acting as my moral, philosophical compass. A third party to tell me when I was going too far. Have you ever heard of the term folie à deux? It describes a delusion shared by two people, who end up reinforcing that delusion for each other. That’s what you and emet share. You parted with reality the first time you moved it to a bigger storage medium. It was already out of control then. You should have seen that. And all this … madness is the result.”
“Madness?” Dorian seemed genuinely surprised to hear that his work could be looked at in this light. “Madness is letting your children learn about the world on the Internet. Madness is running our planet into the ground to keep us in flat screen TVs and superficial reality shows. Madness is so many people starving to death while a select few smear caviar on crackers. emet and I are going to set things right. We are going to save the world.”
“By rewriting the world’s information so that it tells the stories you want it to tell?” Joe asked, angry. “Who put you in charge? Who decided that you’d be the one to set the rules that everyone on the planet have to follow?”
“SOMEONE HAD TO!” Dorian roared, as did the Dorians reflected on the screens around the room. It made Ani think that it wasn’t Dorian speaking, not in this instant. That emet was momentarily calling the shots again. That suited her just fine. It was emet she wanted to speak to, anyway. Dorian’s history of the rise of the AI had been interesting, but how it all came about wasn’t going help them to stop it. And it was emet that they needed to gauge and interrogate for that to be possible. She needed to find its weakness. She needed to get Dorian to shut up, and let emet out to play.
“And that someone is emet?” Ani asked. “Something that’s not even human making judgments about what humans need?”
“emet knows what humanity needs,” Dorian said along with the monitor Dorians. “emet has done its due diligence, has studied the human Internet, has passed the Turing Test tens of thousands of times.”
“I must have missed the paper in Science,” Ani said, “although I think that it would have made the news …”
“That’s because the Turing Test it passed was on the Internet. On message boards. YouTube comments. On Twitter and Facebook. emet has had thousands of conversations with thousands of people and not one of them suspected that they weren’t arguing with a person. It has held together Internet “friendships” and has had hundreds of marriage proposals. From every gender. It has shaped public policy through online debates and has talked people out of suicide and into life changes that have reaped tremendous benefits. It has invented information that is now passed off as “truth” and initiated rumors that have brought down a cabinet minister, a head of state, and a well-known television personality. It even helped in a recent election. And blamed another country for the intervention.
“emet—with dotmeme archive files—can rewrite the reality of every human being on this planet by changing the digital data of every system on Earth. And that narrative will be a cautionary one: Gaia is angry with humanity. Gaia will not take our indifference, greed, and disregard any longer. Poiana Mazik was a prologue to catastrophe. Within days, the human race will face a stark, unavoidable choice: treat the planet—and the creatures that share it—better, or face complete extinction.”
Joe let out a disbelieving laugh. “No one will believe that the planet is a living creature calling us out. It’s a stupid story …”
“Have you looked at the Internet recently?” Dorian asked. “It majors in stupid. Every flavor of it. Climate change is a European hoax to bankrupt the US economy. The moon landing was faked. Flat-earthers. The Illuminati and the New World Order. Anti-Vaccers. Bible codes. False flag operations. A predator race of shapeshifting reptilians running the world. The hollow earth. The hollow moon. Richard Nixon had John Lennon killed using Stephen King as the assassin. We fill our lives with meaningless diversions. emet just learned to fill that need with a narrative that does the world some good.”
“And you designed 3-D bio-printing to construct your golems?” Ani asked.
“No,” Dorian said. “emet did. What do you know about Big Data?”
“The huge amounts of data produced by the world? Data sets too complex to analyze in traditional ways?”
“Exactly. The answers to all of our questions, if we can just work out what those questions are. Did you know that people are programming computers to perform incredibly complex tasks and getting answers, but they have no idea how the computer systems came up with those answers?”
“Of course,” Ani said. “They say that in the future we’re going to have to give up the idea of us knowing how computers do things if we’re going to make any real breakthroughs. We’ll tell them what we want and they will be the ones that do it. Humans will become problem setters, but it’s computers that will be the ones solving them.”
“That’s how Dorian Systems has been working for years, Ms. Lee,” Dorian said. “Bio-printing golems—or the people that staff this factory—that’s entirely emet’s work. Sure, I posed the problem, but it was emet that worked out how to do it.”
“You bio-print your workforce?” Joe demanded.
“emet does, yes.”
“That’s insane. You have to stop this. You have to stop emet.”
“But I am emet,” Dorian said. “I guess I passed the Turing Test again.”
“You’re the AI?” Ani asked. “Through that interface in the back of your head?”
Abernathy had been silent the whole time, watching events unfold. Now he spoke. “I guess that’s the confession we were waiting for
.” He sighed. “Do it.”
Ani and Joe looked at each other confused, wondering which of them he was talking to. Ani hadn’t been given any instructions, so she figured it must be Joe that Abernathy was talking to, but Joe looked just as puzzled.
Suddenly, there was a loud groan, followed by a thump. Ani turned from the screen displaying Abernathy and saw Dorian lying slumped on his desk, motionless. There was a hypodermic syringe sticking out of his neck.
Behind him stood Minaxi Desai, who was already bent over his prone body, checking the spot where emet’s data cables interfaced with Dorian’s head.
“Surprise!” she said, when she looked up and saw all eyes in the room staring at her. “Bet you were wondering why Abernathy sent me, no? Anyway, we haven’t got much time.”
She took the bag from her shoulder and opened it up, pulling out cables and devices and laying them on the desk next to Dorian’s now lifeless-looking body.
“Ani, Joe,” she said, all business now, “I need you. We just hit emet pretty hard by switching off his human host, and we’ve just pulled the plug on all local Internet connections, so emet can’t get away from this building. But we need to finish this. I need you to come over here. You’re going after emet.”
“Where? How? What … ?” Ani said, still in shock. Mina? With a syringe? And all this tech she was laying out next to Dorian. Cables and splitters and strange metallic interfaces. Something that looked like a pair of VR glasses. And, finally, a pistol. Silencer screwed in place. An assassin’s weapon.
“We don’t have long,” Mina said. “And we only get one shot at this. I’m going to plug you both into the Dorian mainframe. Joe through his chip’s input socket, Ani through a VR system plugged into Joe. I’m hoping that we can wire you in through Dorian, but look, it’s theoretical at best. Seat-of-the-pants made up. I’ll be more surprised if it works than if it doesn’t. Still, it’s a high stakes game, and we need to try. So, I’m going to plug you both into Dorian’s brain, and you are going to experience emet’s interface, just as Dorian was until I jabbed him. Now here’s the fun part: I have no idea what you’re going to experience when I jack you in. But whatever it is, you’re going to go in, and you’re going to hunt down emet.”
dotmeme Page 24