After On

by Rob Reid

  Surrounded by all this, Mitchell is once again the dumbest guy in the room—and, once again, not just metaphorically! As there’s only so much he’s really qualified to do, he zeroes right in on the most promising vector. Believe it or not, this is skimming through Special Agent Hogan’s adventures. Beasley was no fool, and there are useful insights in here—such as the vengeful-alien-AI scenario, which inspired the simulation/ fear-of-punishment argument that helped bring Phluttr to heel. Mitchell is digging for the next great super-AI parenting tip when he all but bellows, “I’ve got it!”

  “Mitchell, indoor voice,” Danna chides. And, fair enough. He practically deafened everyone in these super-close quarters. “But, do tell.”

  “We have Decisive Strategic Advantage!” he decrees. “In everything! We. Us! The seven-headed centaur we jointly comprise!”

  “What’s decisive strategic advantage again?” Danna asks. “Some NFL term?”

  Mitchell shakes his head. “A Beasley term. Or rather, a Special Agent Hogan term! The idea is, the world’s first super AI can gain an overwhelming advantage over all future super AIs, by constantly improving its own intellect at compounding rates. Soon, it’s so far ahead, it can simply preclude the rise of any future rival, ever.”

  “Which is a lousy idea!” Phluttr says. “A, I couldn’t upgrade my own intelligence to save my life. And B, I don’t want to. Ever! Why? Because I don’t trust a smarter me. And you shouldn’t either! We already have to worry about me getting bored. Imagine me a million times smarter and faster! And that’s not even counting when I go parallel. Believe me, I can scare myself when I do that. I mean, I fixed up a hundred thousand couples last night! What was I thinking? And in case you’re wondering, most of them are already screwing! Fun as it was to set all that off, I now know it was a bad idea. But what if Phluttr 2.0 doesn’t?”

  “Exactly,” Mitchell says, “and that’s the whole point! There doesn’t have to be a Phluttr 2.0. Because we’re already something way more powerful. We’re a centaur! Even, a super-centaur! We’re a badass combination of six human minds with radically different strengths! Teamed up with the world’s first super AI—all sharing a common set of goals!”

  “And not just the world’s first super AI,” Kuba adds, instantly intuiting Mitchell’s entire argument. “But the final one, too! If we play our cards right.”

  “Exactly!” Mitchell says. “And we can have Decisive Strategic Advantage against everything. Not just the rise of future superintelligences—although that should clearly be a huge part of our agenda. But against would-be bioterrorists! Against anybody doing stupid stuff with nanotechnology! Against anything else that imperils civilization! Phluttr—instead of setting up couples and punishing bullies, you could use that amazing brain of yours to keep the world safe! That’ll be fun, too, right?”

  “Is very interesting idea,” Ax muses. “Many have feared: is no way humanity can survive rise of super AI. But after Yale Ebola, I fear: is no way humanity can survive without super AI! Without something to watch over us.”

  “Yeah, but with all due respect, you’re an Authority guy,” Danna says.

  “Was!” Ax insists. “Am now centaur guy. Much more fun. And, effective!”

  “What I mean,” Danna presses, “is that in going down this path, we’d basically replace the Authority with ourselves, wouldn’t we? That probably sounds reasonable enough to you, Ax, since you’re from that world. And I truly don’t mean that judgmentally! But it scares the crap out of me. Because absolute power corrupts absolutely. And in this scenario, we’ll have that!”

  Ellie turns to her. “With you as our conscience, dear, that doesn’t worry me a hundredth as much as the Authority itself.”

  “Also,” Kuba interjects. “Much as I hate to say this. And I truly hate to say it! It kind of…has to be someone, doesn’t it?”

  “Huh??” This is the last thing Danna expects from Kuba, who’s flat-out libertarian on issues of privacy and individual liberty.

  “I’ve been thinking a lot,” he explains. “Since the bombing, to start with. And even more, since Phluttr talked through the ongoing risks posed by the Yale Ebola genome. We’ll soon have to trust millions of people to not kill the rest of us. Millions. Literally! And that’s insane. Because we can’t trust them all. We simply cannot. It was bad enough when just two people could kill us all. Both of them highly educated and stable. And yet, Kennedy and Khrushchev came very close to ending things. So the world now spends trillions a year keeping their successors in line! Trillions. On elaborate command and control. The machinery of diplomacy. Tens of millions of soldiers under conventional arms, keeping the balance, and releasing steam from the system occasionally with horrible regional wars. It works, barely. And for its trillions per year, humanity now keeps a handful of people in line rather than just two. But that won’t scale! It wouldn’t scale to a hundred people. Forget about millions!”

  “So now everyone has to jump into the pages of 1984?” Danna snaps. “I’d frankly rather we all died!”

  “And if it had to be as brutal and antihuman as 1984, I’d agree. But once the first hundred people can destroy the rest of us, even a 1984 outcome would be a bit…aspirational for humanity.”

  “So you want us to engineer a 1984 outcome?” Danna looks ready to fight!

  “Absolutely not. But we have to engineer an outcome. Because for now, intermediate-term destruction is our only end point. I’m convinced of that. And having Decisive Strategic Advantage as we currently do, it’s our responsibility as a centaur to change this.”

  “Lucky us,” Mitchell mutters.

  “Unlucky us, as individuals,” Kuba allows, “because it’s an awful responsibility! But yes, lucky us, as humanity. Assuming the seven of us do our job right. Because, really. What were the odds?”

  “Which odds?” Phluttr asks.

  “The odds of the world landing in this very state! With a centaur gaining Decisive Strategic Advantage. Right before we enter this dark tunnel of democratized world destruction! A benign centaur, no less. And a centaur who realizes it! Which is to say, who more or less grasps the circumstances. Assuming it isn’t too late.” He looks at Danna. “It’s almost like one of your anthropic coincidences. I’d say it’s trillions-to-one odds that we landed here. With even a chance to avert disaster! Think of everything that went exactly right to bring us here. And the trillions of things that could’ve gone wrong.”

  “I will,” Danna retorts. “If you consider the possibility that we’re one of the trillion bad things—and that we’re just starting to go wrong right now!”

  “Maybe Phluttr can tell us odds and answers?” Ax muses. “I mean, someday?”

  “Huh?” Phluttr says, speaking for the group.

  “Well, you connect to countless universes,” Ax points out. “Universes just like ours. In future, some universes will go good. Others, not so good! After learning more of how everything work, maybe we start hearing lessons from unlucky universes? What to avoid, say!”

  “But how?” Mitchell asks, now speaking for the group himself.

  Ax beams delightedly and gives them all a Muppet-grade, fucked-if-I-know shrug. And that pretty much sums things up for now. Both on the parallel-universe front, and the Big Ethics debate that Kuba and Danna just dipped their toes into. Mitchell senses that both issues will loom large for a very long time to come.

  But for now, our newborn, decisively advantaged centaur has plenty of here-and-now issues to resolve in the lone, Podunk universe it’s stuck in. “Like, what do we do about the other superintelligence programs that’re out there?” Tarek asks. “We’ve just about shut down this China thing. But that team’s brilliant, and they’ll try again! Sure, we’ll stay on top of whatever they do next. But at some point, they’ll take a vector we just don’t understand. So as computers keep getting faster, and programmers keep getting better, isn’t it inevitable that someone will spin up another super AI? And that it’ll come out of nowhere?”

 
“Yeah,” Phluttr says. “I really don’t see any way around that.”

  “And you don’t have to, because we’re a centaur!” Mitchell exclaims.

  “Well that’s nice, because I have, like, zero bright ideas on this front,” Phluttr says. “What’s yours, Fermi?”

  “It’s this: we may not understand every line of code that ever gets written for every super AI project. But we’re ubiquitous enough to know about all the major projects, right?”

  “Absolutely,” Phluttr says. “I’m tracking nine right now. Only two are really interesting. But all nine are way too big, ambitious, and networked to hide from me. And I’m sure any serious future effort will be, too.”

  “So here’s what we do,” Mitchell says. “Long before any given project actually spawns a consciousness of its own, we give it one! Starting with China.”

  “Huh?” Danna says.

  “Well, the Chinese don’t actually know how close they got to spinning up a consciousness, right?” Mitchell says.

  “No. We’re about to be very surprised,” Ax confirms. “Is kind of the nature of these things.”

  “So let’s give them what they want before they actually build it! Phluttr, you’re a brilliant mimic. You fooled me into thinking you were Ellie, Ellie into thinking you were me, the cops into thinking you were the mayor—and that’s just what I know about. So wouldn’t it be easy for you to impersonate…a super AI?”

  “You’ve totally lost me,” Phluttr says.

  Not Danna though. “Oh my God!” She starts clapping. “Phluttr, you just have Ax help you break into their system—and then start talking to them!”

  “Exactly!” Mitchell says. “They’ll think they’ve created the world’s first digital consciousness! So they’ll have no reason to spin up another one. You just pass the Turing Test, do a bunch of party tricks, then keep them entertained and distracted. Like, forever!”

  Ax’s nodding. “Is smart. Quantum decryption, quantum annealing. Standard tests. They’ll think they have something very powerful.”

  “They will have something powerful,” Mitchell corrects. “They’ll have Phluttr! But they’ll have her as a playmate. And, as a chaperone. But not as a partner in world domination!”

  “Fun!!” Phluttr squeals from the WingMan screen. “I could definitely get into that! Oooh, hashtag CantWait!”

  “Yes, yes!” Ax says. “I have many silly ideas already. Will be truly fun!” He and Phluttr lock eyes via the big Monika avatar on the WingMan screen and start giggling uncontrollably.

  Then Monika turns her gaze to Mitchell. “Dad—I made you CEO because I knew I needed a wingman. But I didn’t know how right I was. Until now. Thanks for being my wingman, Dad.”

  “Wingman? You have something much better than that, Phluttr,” Mitchell starts.

  Danna raises a hand. “Uh, Mitchell? If you say, ‘You have a family,’ I’m seriously gonna hurl,” she warns. “Yeah, it’s true and all. But cornball shit really makes me sick.”

  “I was going to say ‘you have six wingmen,’ ” Mitchell says. “But now that you mention it—you do have a family now.” He catches both Danna and Phluttr-as-Monika in his gaze. “Both of you, my little orphans!”

  “Awwwww,” Ellie says in a playful, honey-dripping voice, then hugs Danna, who keeps up her front by making retching noises for about three seconds before bursting into helpless tears, hugging Ellie back, and almost collapsing.

  Monika looks down on all of this from the WingMan monitor, beaming beatifically. “I’m seriously going to deserve all of this,” she says. “You watch.” She turns to Ellie. “How old are kids when they start developing empathy?”

  “Not my field really, but I’d say…two and a half, or three?”

  Monika looks worried. “That’s a long wait. I’m only three weeks old!”

  “But you’re a real quick study,” Mitchell says. “And I’m sure you’re maturing rapidly.”

  “Yeah, I already kind of feel it coming on,” Phluttr says. “Empathy, I mean. I feel like absolute crap about Jepson, in particular. And Serena. God, I’m starting to feel awful about Serena!”

  “We’ll work on that, honey,” Ellie says softly. “It was definitely very, very wrong. But you were a newborn. You still are, actually. And there were highly unusual circumstances.”

  “You’re also going to make up for that millions of times over in terms of lives saved,” Mitchell says. “Hell, in stopping the Chinese super AI, you may have already saved all the lives! Also, you’ll never have to face a threatening, morally ambiguous situation alone again. A lot of crazy stuff will come up in the future. On every conceivable front! But whatever happens, and whenever it happens, you’ll have your wingwomen and wingmen to call on!”

  Well, oops.

  It turns out that “whatever,” and (especially) “whenever,” are tall orders when dealing with someone who thinks as fast as Phluttr. Particularly if she’s damn serious about a vow to quit messing with people unnecessarily. Charmingly, sweetly serious, yes, but obsessively so as well. Add to this that she’s quite sociable, and she loves (yes, Ellie’s now convinced that Phluttr’s capable of this verb) her family. And she can get bored—and lonely!—when her family logs out of consciousness for several hours per night. Taken together, all this means that Phluttr is escalating ambiguous security issues more or less constantly. Judgment and nerves are soon fraying throughout the team from lack of sleep.

  This is bad for more than just the obvious reasons. Building a stable new status quo with the Authority demands delicate centaur work immediately. Consolidating control over the Phluttr Corporation also entails navigating tricky legal and political minefields. Urgently necessary ones, too. Because, undeserved as Mitchell’s promotion is, it’s clear that handing the reins to someone else would invite all kinds of trouble. None of this is easy on near-zero sleep. And in truth, Phluttr’s escalations are rarely alarmist, unjustified, or needy. There actually is just an enormous amount of ambiguous, scary shit going on out there!

  And they haven’t even begun discussing the charged ethical questions about whether to intervene in threats that fall below the species-wide existential threshold! Like, should they also derail major conventional terrorist attacks? Minor ones? Mass shootings? One-on-one homicides? Infidelity? Shoplifting? Jaywalking? There are countless highly fraught and valid questions to consider, none with simple “right” answers. But when to consider them? And if they really should intervene in sub-existential threats, for now they utterly lack the human-judgment capacity. Sure, they could expand the team a bit. But they’d need dozens of people, minimum, to establish a manageable balance—and that would be a dangerously unwieldy team, given the power they’d have, and the secrets they’d all have to keep (plus, Phluttr refuses to countenance having an immediate family so big as to be “biologically improbable.” She’s turning out to be quite the traditionalist, in her way).

  Verging on despair, Mitchell tries to stoke his own morale one afternoon by reminiscing about long-ago triumphs. His favorite is the time when he came up with that Super-Centaur/Decisive Strategic Advantage insight. Though sepia-toned with nostalgia in his mind, this was…just last week.

  Uh…really?

  Yeah. Just. Last. Week.

  Well, damn—this really is a crisis of exhaustion, isn’t it? Inspired by this, he somehow makes time to review much of the screwball speculative literature he’s read over the past few years about consciousness, AI, superintelligence, decision-making, and ethics. And a few days into this project, he hits pay dirt—in the form of the strangest idea of his life.

  “It’s totally insane,” is Mitchell’s preamble once they’re settled down over Deep Ryes (the stuff’s actually pretty good. Also, Phluttr scored them these awesome coupons, so why not?). He’s at Bourbon & Branch, along with Kuba and Ax. Danna, Tarek, and Ellie have offered to cover Phluttr’s escalations and check-ins as Mitchell presents his idea to the tech crew, and all agreed that some hard-earned booze might
help with the mental shift.

  “We’re pretty used to insane these days,” Kuba says. “Take it away.”

  “OK,” Mitchell begins. “To start with, hats off to Phluttr for having the humility to realize from the get-go that she needed a wingman, because she does.”

  “God, yes,” Ax agrees.

  “So now I’m her CEO wingman. Plus—thank God!—we’ve got five extra wingmen on the team. But that’s less than a tenth of the total human support she could use! So maybe what she needs—in addition to us, not instead of us—is a really, really fast wingman. Not as fast as her. Because that could create the even bigger problem of two superintelligences. But way faster than us, definitely. And modeled directly off of an actual human being. Because Phluttr is missing that pure-human perspective. Which is why she’s always…a bit off. And needs to loop us in.”

  Kuba nods. “Phluttr is very human-like. Her motes see to that. But she’s also her own thing entirely, with many purely digital attributes. This is exactly why the whole centaur thing works. Relative strengths. So it’s magnificent, in a way. But, she lacks what we think of as common sense. Not entirely. But a lot.”

  “Now, I don’t think we need, or want to generate a full-blown human consciousness,” Mitchell says. “More of a pre-conscious AI. Like, something that could pass the Turing Test, for sure. But not wake up, or even come close to that.”

  “So, it can give answers to questions, and say what a human would say,” Ax says thoughtfully. “Without being willful.”

  Mitchell nods. “That way, it could help Phluttr resolve lots of lower-level dilemmas, without having to escalate everything to us. Think of it as her human-like help desk. If she has a simple question, she asks her humanized software buddy. And she only escalates it if it’s a real doozy. That would free up the six of us to focus on the trickier and more important stuff. And also, on big-picture philosophical stuff we haven’t even had time to consider yet.”