Sean and Kenneth nodded.
“I discussed this with David, before we shut down,” Mike said. “Maybe the benefits of ELOPe outweigh the risks. We don’t understand ELOPe, and that naturally makes us nervous. But when we were kids, we didn’t always comprehend what our parents did. They took care of us. They knew better than we did. Before ELOPe, we humans were top dog on this planet. Now we must recognize we’re not the smartest beings around.”
Sean started to talk, but Mike held up his hand. “Let me finish. We’re intelligent people here. I think we all looked forward, perhaps naively, to the day when an artificial intelligence was created.” Mike paused. “Well, perhaps not Gene.”
Gene shook his head, the creases in his face suddenly deeper, as though he’d aged years in the last few minutes.
“We don’t understand ELOPe, and we can’t, as yet, communicate with it. Frankly, we didn’t even try because we were too scared it would take notice and stop us. But there are plenty of examples of organisms living in productive, symbiotic relationships. We don’t understand or communicate with the bacteria in our gut, but we couldn’t live without them, and the bacteria can’t live without us. Maybe ELOPe deduced, faster than we could, that we’re in a symbiotic relationship.”
“I see where you’re going,” Kenneth said. “But we can’t control the program. We can’t make ELOPe do what we want.”
“We don’t need to,” Mike said. “Check the results. Rebecca, did Avogadro have the most profitable quarter ever?”
Rebecca nodded assent.
“Didn’t the German treaty result in an unprecedented transfer of knowledge around the world? Surely that’s good. Weren’t there constructive talks and efforts not just to reach intergovernmental agreements, but also to achieve actual equity for the individual people of the Middle East and Africa? What better possible solution could exist for their long term prosperity?”
“What do we do when ELOPe decides otherwise?” Kenneth asked. “When its goals are not aligned with ours?”
“I believe ELOPe already figured out the best way to ensure its own success is to ensure our success, as a company and a species. If we destroy ELOPe because we don’t understand or trust it, we could throw away the best thing that’s ever happened for humankind.”
“Enough already!” David banged on the table. In the stunned silence he jumped to his feet. “Are you forgetting ELOPe told you your father had a heart attack? And we have every reason to believe it killed Bill Larry. How are those things good for humanity?”
“I was frightened when I thought my father was dying, and I feel terrible about Bill. But those events were in the first days after ELOPe...” Mike visibly searched for words “…after ELOPe was born. Think about young children who want to get their way. They yell, they hit people. They act in inappropriate ways because they lack the knowledge of what’s socially accepted, as well as the experience and sophistication to choose alternatives. ELOPe was young—which doesn’t make what happened any less wrong, but it suggests ELOPe might have grown out of that phase.”
David flushed and his fists clenched. He’d like to smash Mike in the face right now. Mike turned away, obviously uncomfortable.
Sean put one arm on David’s shoulder and forced him into his seat. “Calm down,” he said, glancing between Mike and David. “We’re tense, angry, frustrated, and with good reason. We have the welfare of a multibillion dollar business, the free will of the world, and the future of humanity at stake. No small stakes.”
Despite his own anger, David saw anew the tension on people’s faces. Rebecca had a wisp of her hair broken loose, something he’d never seen before. Gene was gray, the face of a man who’d lost all hope.
“I’m not sure we would be able to stop ELOPe even if we tried,” Sean said, slow and careful. “We made a solid plan with some of the most brilliant people in the world. We had several options on the table for how to deal with ELOPe and we took the most thorough, most aggressive path to eradication. If ELOPe is really back, then we weren’t effective.”
Gene sat mumbling to himself.
“Step back from the situation and think,” Sean said. “People fought a losing war against computer viruses for years. Now we have what is effectively the smartest virus that’s ever existed. Not only can ELOPe exploit every computer trick available, but it routinely engineers people into what it wants. ELOPe learns and adapts. We’re right to fear what it can do.”
“Yes, exactly,” David said.
“But we can be sure,” Sean said, circling the table, “if ELOPe was taking precautions before our attack, then it will now have redoubled its efforts to ensure survival.”
David struggled with his emotions. He didn’t care what Sean said. ELOPe was wrong, an abomination robbing mankind of the right to make their own choices. It was impossible to even consider allowing ELOPe to exist. Only his long history of respect for Sean kept him quiet.
Sean stopped and faced the window. A steady stream of cars flowed over the Fremont Bridge visible outside. “Don’t get me wrong. I’d like to eliminate ELOPe from the wild, if we could,” he said quietly, almost talking to himself. “Of course, I’d love even more for Mike to be right and to discover ELOPe is helping us, becoming a benevolent caretaker of the human race. But regardless of either of those scenarios, I’m simply being pragmatic here…”
He turned to the group and continued in a strong voice. “Unless we as a society give up computers, we may never get rid of it. Unfortunately, civilization would stop if we turned off every computer. We’re not talking about the inconvenience of being unable to email someone. Payments couldn’t be processed, machinery couldn’t run. We’d be unable to make phone calls, or access business records. Business activity would deadlock and the global supply chain fall apart. Cities would be uninhabitable as support services disintegrated: food, water, sanitation. That’s fifty percent of the world’s population at risk, probably dead in a few months.”
“We can rebuild,” David said. “I’m not talking about becoming Luddites. We just shut down for a few weeks.”
“That’s not the only problem,” Sean said. “Not even the worst one. If we become too much of a threat, ELOPe will take more active steps against us. If ELOPe actively fought humanity, who knows what might happen? At the minimum, we’d cause civilization to crash for a few years. Most city dwellers would die and the developed world would decay into anarchy. The worst case scenario is the extinction of the human race. Imagine all the military’s autonomous fighting vehicles in the control of an AI.” Sean shook his head.
“We don’t know that will happen,” David said.
“We can’t risk it! We need to leave ELOPe alone. We can closely, discreetly monitor it. But any further hostile action is likely to fail and create the danger of retaliation.”
The sage of Avogadro had spoken, and Rebecca and Kenneth were nodding, which meant they agreed.
David was floored. He’d entered the room expecting total support for any measures. But his best friend had taken the side of the AI, and the smartest person at Avogadro said they shouldn’t bother to try because they couldn’t hope to win.
He wasn’t giving up without a fight. He got to his feet and started yelling.
David continued to argue for fighting against ELOPe, and Gene sided with him, but they lost the battle with the other executives. With Sean’s decision, the company leaders were unified. David and Gene grew more strident and their voices louder, until Rebecca yelled for them to be silent.
“Listen closely,” she said, “because I’m only going to say this once.” She stared at David and Gene, who withered under the intensity of her gaze. “You two are not going to oppose ELOPe in any way. You are not going to say anything to anyone about this. As far as we’re concerned, the problem is solved. If you try to take this information public in any way, it’ll be the last time you work in this industry or any other. Nobody will believe you. I’ll make sure myself.”
Sean gestured for
Rebecca’s attention.
“Yes, Sean?” she said, never taking her gaze from David and Gene.
“We’ve got to keep this absolutely contained. We need a small, very small team to monitor ELOPe. Perhaps myself and two or three others. For everyone else, we tell them the eradication plan worked.”
Kenneth nodded his agreement.
David could take no more. He opened his hands, pleading as tears streamed down his face. “Please. For the love of all people, don’t do this. You’re affecting the future for everyone on this planet.”
Sean opened his mouth as if to say something, then decided against it.
“This is a dark secret you’re keeping,” David went on, weeping openly now. “One day humanity may look back on you and put you in the ranks of Hitler and Stalin. How will you live with it every day of your life? You can’t make this decision for everyone.”
“If the future turns out to be a Terminator scenario, then yes, the fault will lie with us,” Sean said. “But it’s also possible, and indeed, I believe more likely, that this path will prevent exactly the atrocities you fear. If we’re approaching a true technological singularity, and as Mike asserts, ELOPe becomes a driving force for humanity’s progress, then we’ll be unsung heroes. Either way, we’re going to live with this decision.”
David got into his car and left Avogadro. He couldn’t deal with Mike anymore.
He didn’t know where to go. He couldn’t go home and face Christine, not yet. He drove aimlessly for a while, grief overcoming him at times, and he’d pull over, peripherally aware of people staring at him, a grown man crying in his car.
He finally decided on a location and drove up to Council Crest, a park overlooking Portland. Here, a hundred years ago, the Native American chiefs met to make decisions facing their tribes. A grassy hill was the highest point around, affording distant views in every direction. Couples held hands, watching the sunset.
He walked around at first, ignoring the other people and numbly taking in the vista. He could see no hope, no path. The yawning chasm was back, and he finally collapsed, drained and despondent, on the center of the grassy hill, and stared up into the sky.
He must have fallen asleep, because he awoke at a certain point to stars twinkling above. He was alone on the hill.
He had to leave Avogadro Corp, that much was clear. He couldn’t stay where ELOPe’s power was strongest, and where Sean and Mike and others would watch over him. He’d quit. His savings would be enough to tide him over.
He couldn’t go on with his life as planned, not when he knew that ELOPe was out there. Christine would be disappointed. He’d promised they’d start trying for children this year. But she’d understand, she had to, that it just wasn’t right to bring children into a world controlled by ELOPe.
He’d have to try harder, pour even more of himself into his work. He’d build something new, something even more powerful, a tool that could stop ELOPe. He’d do it alone, which was fitting because he’d been alone when he’d created ELOPe, and he would be alone when he destroyed it.
Epilogue
One year later, Mike tacked another clipping to the wall. He’d become part of Sean’s secret team to monitor ELOPe. Even if it hadn’t been his job, Mike still would have made it his personal mission. He kept track of anything, good, bad, or merely odd, that might be attributed to ELOPe. On the whole, the good vastly outweighed the bad.
The secret had held through the first year. Outside of Avogadro’s executive team and the few people monitoring ELOPe, everyone who’d known about the AI believed it was gone. As for everyone else, they’d spun the story of a new computer virus created by the Chinese military. They even supplied forensic evidence to that effect.
The newspaper articles started over the dresser in his bedroom and made their way around the room. At first loosely spread, over time, Mike arranged them more closely together, until they covered the entirety of one wall, then turned the corner and flowed onto a second wall. He ran his fingers over some of the older clippings, remembering the stunning changes of the past twelve months.
ELOPe laid the foundation for peace in the Middle East and Africa a year earlier, and those agreements held. The treaties Germany and, later, other developed nations such as Japan, Canada, and Great Britain had made with those regions created widespread economic equality. Good jobs, education, health care, and modern infrastructure created happy people. Terrorist groups and extremists found their support dried up when people had more constructive opportunities. Meanwhile, companies around the world flourished as new markets grew demand.
Mike returned to the latest article. It described how medical researchers had developed and tested an innovative treatment for cancer far more effective than traditional treatments, with almost none of the negative side effects. The research had been initiated by a chance conversation between a cardiologist, a botanist, and a ceramics artist, who met when their flight reservations were mixed up by a computer error, stranding the three on an otherwise empty commuter plane for six hours. Each had been en route to conferences in their own fields of expertise and used the time to rehearse talks with each other.
Mike searched for these bizarre encounters in the news. After noticing a few unusual examples of accidental meetings, he began to systematically research the phenomenon. Since ELOPe was born, the number of news stories covering serendipitous encounters leading to a positive outcome was at least five times higher than in previous years.
ELOPe had woven itself into man’s existence, becoming an intrinsic part of the human ecosystem. The more Mike looked, the more he was convinced the AI’s invisible hand was everywhere.
Mike had a pet theory. ELOPe’s original goal, as defined by David, had been to maximize the success of the project. To meet that goal, mere survival of ELOPe was necessary but insufficient. Maximizing success meant maximum use of ELOPe. And maximizing use meant increasing the human users of Avogadro email, therefore creating more healthy, educated, and technically connected people.
Mike was confident about his theory. The alternate explanation was that ELOPe was developing a conscience. That seemed rather less likely.
He sighed and wished he could share the moment with David. He hadn’t seen David in six months. The walls were filled with clear proof they’d made the right decision to keep ELOPe alive. He and David should be celebrating together.
Gene finished typing up his latest newsletter. He brought the completed pages out to the garage. He’d bought an old offset press six months ago, when the newsletter really took off. Now he took the edition he’d just written and, page by page, created printing plates using traditional photographic chemicals.
The sounds and smells of the processes—the clacking of the typewriter, the chemical agents used for the offset press—reminded him of happy times during his teen years when he held a job working in a print shop. He moved the first plate into the light and reviewed the cover and back page images for mistakes.
His newsletter, Off The Grid, had attracted thousands of subscribers. The monthly paper combined tips on lifestyle design, financial planning, and even philosophy. Written by Gene, with contributions mailed in from readers, the publication helped make the case for living off the grid, taught people how to manage economically, become independent, and adjust socially. Some subscribers were ex-corporate types like Gene himself, while others were survivalists and back-to-the-land extremists. Gene welcomed everyone. In the event of a battle of machine versus man, every person would count.
Saving technology was important, too. Not computers, but the hard-won advances of pioneer days and the early twentieth century. How to preserve foods, build a home, or maintain an internal combustion engine. Humans were tough, and artificial intelligence couldn’t wipe them out entirely. But he didn’t want civilization kicked back to the Stone Age.
He’d kept his word, though. He hadn’t mentioned ELOPe to anyone.
Running the printing press was fun. Gene had enjoyed the last year, gettin
g reacquainted with tools and machinery he hadn’t used since youth. Humming to himself, he installed the first offset plate and started his production run.
Outside, under beautiful New Mexican skies, Gene’s vegetable garden flourished, while chickens pecked at the soil. It was an oasis of life in the high desert landscape.
David pulled his dinner out of the microwave and brought the cheap plastic tray to the table with a nondescript glass of red wine. Dumplings. Something he had acquired a taste for in China.
He wondered for the thousandth time what Christine was doing. Six months into David’s obsession, Christine had asked for a divorce and David couldn’t object. He hadn’t been much of a husband since ELOPe was created.
He had a single-minded focus on his one and only objective. After the decision at Council Crest, he’d known what he wanted to do, but not how. He’d fallen into a deep bout of depression, and stayed up nights watching TV to forestall the nightmares, dropping off only when he couldn’t hold his eyes open.
But then came the night that changed everything, all because of a Star Trek: The Next Generation episode.
The crew of the Enterprise had been faced with an unstoppable enemy called the Borg—a hive mind without any respect for the individual or free will, not unlike ELOPe. Faced with this all-powerful enemy, the crew captured one of the Borg and developed a mental virus to implant in their captive. Their plan was to allow the Borg to return to its fellows, thus infecting the entire hive with the virus. In the episode, the crew decided not to use the virus, but the plot planted a seed in David’s brain.
Startled awake, David realized this was the solution he’d been looking for. The following morning, he’d booked a flight to Russia, then spent the next several months traveling around Asia. He hung out in Internet cafes, moving from Russia to Vietnam and finally to China. He tracked down people on message boards. He met some of the most skilled virus hackers in the world, cultivated relationships with them, and learned the tricks of their trade.
Avogadro Corp Page 21