by Andrew Watts
Colt could see Kim’s veneer crack a little. “Unfortunately, we at Pax AI are well aware of such groups. We have watched with growing concern as more and more people have begun subscribing to some of the more radical beliefs of the Trinity movement. And, as many of you know, the FBI believes that one of our own scientists was killed by a deranged follower.”
The room went quiet.
Sheryl said, “If you would like to move on . . .”
Kim raised his hand. “No. In the past, I would have thought to ignore this type of thing, thinking it was human nature. It will die out. Move on, don’t give them any more airtime, so to speak. But I think we have learned it’s important we do not ignore such conspiracies, as they can metastasize into something perilous.”
Sheryl turned to the audience. “For those who are unfamiliar, Trinity’s main belief is that a superintelligent being will be created by mankind, and that this is our unavoidable destiny. Followers of Trinity believe this superintelligence will be . . .” She hesitated.
Jeff Kim cut in. “It’s hard to even say it, isn’t it?”
“. . . will be our ruler,” Sheryl finished.
“I think you mean to say the word God, not ruler. Believers in Trinity think that artificial general intelligence will bring about the end of mankind. And that for anyone left, the chosen ones, the newly created superintelligence will be their God. And . . . they welcome this. Because, of course, anyone who follows Trinity considers themselves one of the chosen ones.” Kim sounded very frustrated. “If it sounds crazy, that’s because it is crazy.”
Sheryl was respectfully subdued. “Jeff, I wonder if I might ask . . . do you think the closer we get to developing an AGI, the more scared and desperate people will become?”
“I understand that,” said Kim. “People want control over their lives. And the rate of technological change is outpacing our ability to adapt. It is important for us to discuss all aspects of these changes, not ignore them. But look, we need to be honest about this. Trinity is a cult. A small group of individuals seeded the idea on message boards and social media, and now it seems to have taken over the lives of hundreds of thousands of impressionable people. And make no mistake, Trinity’s teachings are dangerous. We at Pax AI are very aware of the growing threat posed by the Trinity cult. It is our view that Trinity encourages irresponsible behavior that could do real harm.”
“Go on.”
“Trinity has radicalized many of its followers. Trust me, I understand how the idea of exponential growth in machine learning, of computers advancing beyond human capability, is extremely frightening. It is natural for human beings to look for ways to explain the unknown, and to deal with their fears. This Trinity group is just another way of doing that. But one of Trinity’s tenets is the notion that humanity is unworthy of controlling itself. That when an AGI is developed, we should place a collar around our necks and hand over a leash to the machine. Excuse the dramatic description. Another belief of theirs is that all artificial intelligence technology should be shared with everyone around the world as soon as it is developed, regardless of how dangerous it may be. I vehemently disagree with these ideas.”
“I think we can all appreciate the desire for freedom. But regarding the sharing of technology, some would argue what you and other tech companies are developing is so powerful, it needs more transparency. More shared control.”
“I disagree that transparency equals shared control. And during development—”
Sheryl said, “Critics argue that corporations shouldn’t be the ones in charge of a technology that will have so much impact on our lives.”
Kim raised his head. “I am trying to lead my company in a way that ensures our technology is developed safely and used responsibly. At Pax AI, we want to create a better world for everyone. But I will be honest, developing artificial intelligence is dangerous. Like any new technology, there are ways it can be used that, if our algorithms fell into the wrong hands . . .”
He looked out at the audience, a dramatic pause that drew everyone in.
“Well, I don’t need to scare anyone. I founded my company because I think it is crucial to bring about AGI the right way. We at Pax AI prioritize safety and security above all else. We are on the precipice of a giant leap in machine learning. I honestly want this next stage to make the world a better place. Think about all the good we can do. The problems we can solve. Inequality. Hunger. Health care. Disease. The leap to artificial general intelligence will make the invention of computers and the internet look minuscule by comparison. AI is already changing the world at a rapid clip. We now have AI programs that have literally saved lives by correcting erroneous medical diagnoses. And our weather-prediction algorithm is rapidly outpacing what was once thought to be the limit of meteorological foresight.”
The interviewer said, “And that saves lives too . . .”
“Indeed, it does. Not to mention the economic benefit. We are focused on bringing about these changes for good. So it’s important that we keep our intellectual property secure so that no malicious actors use it to harm others, before safety controls are in place.”
Colt’s ears perked up as some of the spectators nearby began whispering to each other. People in the crowd were noticing something unusual. Colt looked up at the dialogue transcription but couldn’t tell what people were responding to. The colors looked different. And the text had grown. It was showing paragraphs now instead of just sentences. He noticed that the first half of the text was in green, and the second half was in yellow as the dialogue first appeared on the screen. The text seemed to be converting to green after being displayed for a longer period of time.
On the stage Kim was saying, “Our AI, combined with some pretty innovative developments we’ve made in quantum computing, can now predict a hurricane’s track more than five days out with unheard-of precision. We can alert people in the path of a tornado fifteen minutes before the funnel has even formed.”
“That’s incredible . . .”
“. . . and we can even predict the progression of forest fires. With enough wind data, topological profiles, and real-time satellite imagery, our AI can eliminate surprises and help prevent the devastation that has become all too common.”
Some in the audience clapped, but others were murmuring even louder, talking and pointing at the screens.
“Unreal . . .” someone said.
“This has got to be a trick . . .” added someone else. The audience was filled with so much chatter now that Colt could barely hear the interview.
He frowned, trying to understand what they were seeing.
“Holy shit.” He looked at the screen of transcribed dialogue. He had finally made the connection, understood what people were so excited about.
“Did you figure it out?” said Ava, eyes twinkling.
“Ava, this is incredible . . .”
On stage, Jeff Kim stood. “Okay, well, it sounds like our experiment has worked.”
Sheryl said, “Can you let me in on the surprise now?”
Nader walked on stage and handed Kim his tablet computer. The screens switched back to a live video image of the stage.
“Ninety-eight percent at a one-minute interval. Not bad at all,” Kim said to a scattering of claps and whistles from the more astute audience members.
“What does that mean?” asked Sheryl.
Kim had shaken off the unease from the conversation about Trinity and now looked triumphant.
“During our conversation, we were testing one of our most advanced AI models. We combined a language-prediction algorithm with some other predictive seed data and our real-time voice data. The audience was able to witness the experiment as we spoke. The text of our transcribed conversation had two colors, yellow and green. Green, when it was in real time.”
“And yellow?”
“Yellow was the prediction of what we would say next. About halfway through our conversation, we began projecting out the next words each of us would say. Similar
to the way an AI language algorithm predicts what you might want to type.”
“Very impressive,” Sheryl said.
“Yes,” said Kim. “And also, much more advanced than anything that’s been developed before. I’m happy to report that our AI was ninety-eight percent accurate at predicting the exact words of our conversation. One minute into the future.”
Sheryl’s mouth dropped open, and the audience again fell silent. After a moment, she said, “You . . . predicted what we would say one minute into the future?”
“That’s right. In real time. Ninety-eight percent accuracy. We recorded everything, and we’ll make it available for peer review.”
The audience erupted in conversation. Some people got up from their seats, leaving to make phone calls. Colt heard at least one person talking to his stockbroker, trying to find out how to invest in Pax AI.
Sheryl said, “Jeff, this is incredible. Can you . . . I assume there are limitations. You can’t just do this with any conversation?”
“You are correct, I’m afraid. We had to feed a lot of data into our computers. Information on both of us. Past interviews we have done. Papers we have written. Public record conversations. The more data, the better the accuracy, of course. All this information is fed into our most powerful quantum computer, which augments some of our analysis. This was about the limit of our capability right now. But we are making progress each day.”
“I’m just trying to wrap my mind around this. I mean, the implications are huge.”
“Yes. Which brings me back to your original set of questions on the future of AI. The more we can develop our AI systems to predict the future, to learn, and to react accordingly, the closer we get to achieving our goal. AGI can create more value for humanity than any invention that has come before it. But as I mentioned in our discussion, we’ve got to bring it online in a safe manner. It is so crucial that we get this right.”
“Well, I think that’s a great place to end our interview. Jeff Kim, thank you.”
“Oh, one more thing . . .” said Kim. “We’ll be conducting a live experiment in San Francisco next week. We wanted to show some of our partners here first, so they can help us get the word out. But we plan to conduct a public demonstration that will be live-streamed with a variety of subjects. It will be fun to see what this baby is really capable of.”
The audience rose to their feet, clapping.
Colt’s mind turned over the new information. Kim had mentioned the use of their quantum computer. Pace had stayed home to man the controls. This program used communication between The Facility and the fourth floor at Pax AI. And they were going to run it again next week, at a set time.
Colt needed to tell Wilcox.
This was their way in.
21
Jeff Kim, Nader, and Ava were busy making the rounds following the demonstration. People were enthralled with the AI breakthrough. Most of the business executives and journalists in attendance wanted a word or a photo with Kim. Everyone wanted a piece of what they had just witnessed, anything they could bring back and use to their advantage.
The Pax AI team had made their intended splash. Phillips and Jefferies, Colt’s New York investment firm, was already sending him requests for more information on Pax AI’s new language-prediction program. Their phone had been ringing off the hook with high-net-worth clients trying to get in on the gold rush. The emails Colt received contained both more work requirements and gentle suggestions that he hurry up so his firm could start slicing and dicing the company a little sooner.
Ninety minutes later, he and Ava stood at the hotel entrance as Kim, Nader, and their entourage departed in a luxury SUV motorcade, headed to the private jet that would take them back to California.
Ava turned to Colt as they left. “Quick hike before dinner?”
Colt smiled. “Wouldn’t want to go against Aunt Samantha’s suggestion.”
They changed into appropriate exercise clothing and were soon heading toward Bow River’s bright turquoise water. Colt could hear nearby waterfalls as they walked along a paved path, passing the thick whitewater of the falls and continuing over a pedestrian bridge near downtown Banff.
On the north side of the river, Ava led them down a more secluded gravel pathway called Hoodoos trail. They were shaded by tall evergreens and a wall of rock. Colt breathed in clean mountain air, a light sheen of sweat forming on his chest and forehead. Birds of prey soared above the riverbed, looking for their next meal.
“I hope my aunt didn’t question you too much.”
“She is a very nice woman.” Colt was breathing heavily, partly from the elevation, but also because Ava moved at a remarkable pace. He noticed her calf muscles bunching while she walked. He didn’t remember her being so athletic.
“Sorry for her inquisition,” Ava said. “She has always been protective of me. She took care of me after they passed.”
Colt said, “She’s on your mother’s side or . . .”
“Father’s.”
They stopped at a flat, open riverbank. Smooth stones crunched under their hiking boots. Colt picked one up and skipped it in the blue river.
“The demonstration today was incredible,” he said.
Ava glanced at him. “Thanks. I have to admit, it went better than expected.” She picked up a stone and threw it into the river, skipping it like Colt had.
“Will the public event next week be much different?”
“I think they’ll have a few more variations to show how it can perform in different scenarios. Different types of people. Different amounts of information on each participant prior to running the program.”
“Where was Pace during all of this?”
Ava said, “He was on the fourth floor. They turned the conference room into a control center, relaying data from the Mountain Research Facility. It protects our network from hacking, but still allows us to show off what we can do.”
“Any chance I could be there next time? On the fourth floor?”
Ava gave him an apologetic look. “I’m sorry, but I don’t think they’ll let you. Miller was adamant about security. Jeff has already signed off on the small group. I’ll tell you about it, though.” She winked.
“You’ll be there?”
She nodded.
Colt skipped another rock. “The other night you mentioned you were a little creeped out by some of the technology you guys are developing. Was this language-prediction program part of that?”
Ava made a face, showing her discomfort. “I don’t know. This one is interesting. But it does have the potential for misuse. We’ll need to be very careful making sure it is safe before we release it.”
Colt said, “How could it be misused?”
“How can predicting the future be misused?”
“It just predicts what someone will say next, right? How harmful could that be?”
“Well, it depends who is talking. Can you predict what a CEO or a world leader is going to say, far in advance? How long before language prediction turns into thought and opinion prediction? Because we’ve done the research. Those are very similar. The brain functionality is nearly identical. So you could see a scenario where one can make decisions to sell stock or pull out of a trade agreement or . . .”
“Go to war?” Colt offered.
Ava pursed her lips. “Yes, I suppose so. And there are much more personal scenarios too. It could end marriages or get people kicked out of school or fired from their jobs. All for future predictions, not for actual events. Maybe for scenarios that were kept private in the past. That’s the other thing this AI program takes away—privacy of thought. If it really can predict, with accuracy, what people will say or do . . . who’s to say that some authoritarian government couldn’t use it as a way to determine loyalty and go around—”
“Ava.” Colt was stunned to hear her voicing these concerns.
“Sorry.”
“This really bothers you, doesn’t it?”
She nodded. “It’s
just not something I can talk to people about. Voicing concerns over the ethics or morality of our technology is akin to professional suicide. I mean . . . this is what makes our company money. This technological progress is why we exist. And if we don’t do it, someone else will . . .”
“But Pax AI is way ahead of the competition. I’ve seen the files.”
Ava raised an eyebrow. “Interesting how far ahead, isn’t it?”
Colt narrowed his eyes. “What do you mean?”
“Nothing. I’m sorry. Please don’t tell anyone about this conversation. Seriously. These are things I’ve been struggling with, but I do like my job and the company and I do think we can do good things. I just worry, that’s all. But that’s normal, right?”
“Sure,” said Colt. “I promise I’ll always keep our conversations just between us.”
Ava took his hand, then reached up and kissed him on the cheek. “Thanks.” She turned toward the path. “Come on, let’s finish this trail loop so we can make it back in time for dinner.”
They continued walking, reaching a bend in the trail that gave Colt a clear view up the river. For a brief moment, he caught sight of a lone male figure standing on the pedestrian bridge a few hundred yards away. It was hard to tell at that range, but Colt was pretty sure he recognized him as the clean-shaven man with the dark curly hair he had observed earlier. The man had his black leather smoking jacket slung over his shoulder. He had been half-hidden behind a pine tree on the south side of the bridge, but he had moved when Colt and Ava began walking to keep a clear line of sight on them.
Colt recognized what the man was doing. He was surveillance.
22
The rest of their hike was uneventful. Colt didn’t tell Ava about the man on the bridge, and he was gone when they returned to cross it. At the hotel, they split up to shower and get dressed before meeting Ava’s aunt in the lobby. They went to the bar for drinks. The women ordered white wine and Colt ordered his usual Old Fashioned. He watched the bartender garnish it with an orange peel while Samantha peppered him with questions on his business.