by Ramez Naam
The glass doors parted for him and he paused.
“Wait with the car. I’ll be an hour, perhaps two.” Then he strode on as his driver bowed to his retreating back.
Chen passed through the metal detectors and T-rays, waved his ID, and placed his eye before the retinal scanner. The elevator door opened, and he stepped in. Five floors down, below the main building, he stepped out, and into the Secure Computing Center.
The armed guard nodded at him. Chen ignored the man, and made his way across the facility to the entrance to the PICC – the Physically Isolated Computing Center.
Ling closed her eyes and followed her father’s slate and phone as they made their way down into the earth, deep below the official buildings of Jiao Tong. The devices accessed the local network inside the Secure Computing Center and formed a tunnel back to her. She reached her mind through that tunnel now, gently stroked the flow of data inside the facility, parsing, absorbing, searching. Her mother was here, trapped, cut off from the outside world, cut off from Ling. She would find her.
Chen crossed the facility. Men and women stopped work and bowed their heads respectfully as he passed. He was Chen Pang, after all, the architect of China’s explosive surge in quantum computing. He was a figure of awe to them. If only they knew.
Li-hua saw him coming. His assistant rose to her feet – a homely woman, too short, too pudgy. She bowed her head to him. “Honored Professor,” she said, “the tests you requested…”
Chen waved her away, kept walking. Yes, yes, she’d done the tests. But he needed to see them again.
He kept walking, past all the bowing underlings. The resentment rose in him again. The envy. That his greatest accomplishment was to serve as the secret scribe for his wife. That she was the true discoverer of so much.
Still, it had its rewards.
“Ling, your break time is over. Time for your next lesson.”
Ling clenched her small fists.
“Ling.” Her tutor’s voice rose slightly.
Ling forced herself to smile, forced herself to turn back to the human, the way her mother had taught her, forced herself to say the insipid words.
“May I please have a few minutes more, teacher? I like to watch the rain.”
“Well.” The tutor sounded surprised. “Since you asked so politely, you may.”
Chen surrendered his electronics at the next checkpoint. Then the security man slowly and thoroughly wanded him, looking not for weapons, but for any device that could possibly carry data in or out of the PICC.
The guard finally declared him clean, and Chen stepped forward and into the cavernous elevator. The doors closed behind him, and the elevator started its descent through one thousand meters of bedrock and towards the mad software entity that was all that remained of his dead wife.
Ling sifted through exabytes of data. Cryptographic libraries. High resolution satellite imagery. Whole brain scans. Genome sequences. None of it was her mother.
She looked for maps, physical maps, network maps. She found them. The network topology told her little. Nothing obviously fit the description of the quantum cluster her mother existed in. The physical blueprints of the building were no more helpful. Multiple data centers existed here, but their functions weren’t clear.
Ling kept searching. She would find her mother here.
The room-sized elevator took Chen down through the rock beneath Shanghai. A lit sign declared that the current PICC status was
ISOLATION IN EFFECT
All this was a precaution. Computer scientists, philosophers, futurists, writers of speculative fiction – they’d all written about the dangers of runaway superintelligence. If humanity ever created a being of radically increased mental capabilities, it placed itself at grave risk. That new being could be benevolent, of course. That would be the hope. Or it could be malicious, or simply indifferent to humans. It could seek to change the world in ways that it saw as improvements, but which were incompatible with the interests of its creators.
A superintelligent being might also be able to improve on itself, reaching into its own structure and finding ways to optimize them, to make itself smarter than its creators could have, with no obvious end in sight.
And for that reason, Su-Yong’s ability to edit herself was limited to superficial layers only.
Chen himself doubted the risk of runaway self-improvement. Intelligence showed diminishing returns. Just as a single human could not design a human-level intelligence from scratch, no superhuman creature could possibly design a creature of its own intelligence or greater. Oh, it might be able to make improvements on the methods used by its creators, and get some boost, but without collaborators, without access to new hardware, the improvements would level off.
And so he’d stretched the rules, hidden a few upgrades of his wife’s design in with more prosaic maintenance, made the case for upgraded quantum cores. All for sensible reasons, of course. All so she could produce more work of value to the Science Ministry and the State and humanity. All for the greater good.
Regardless, in this world, this world where everything was linked, where data ruled all, where cryptographic codes had replaced physical locks on the world’s wealth, on its infrastructure, on its weapons… In this world a being able to process information more rapidly than humans was the ultimate threat.
It was for such reasons that the Copenhagen Accords prohibited any attempt to create a non-human self-aware being. And for the same reasons, Chen’s own government, in sponsoring the creation of exactly such a dangerous and illegal being, had taken extreme precautions that it could be physically isolated, cut off from the outside world, and destroyed remotely if necessary.
The elevator clanged to a halt. Here, at the bottom, where no wireless transmission reached, three physical data lines connected to the outside world. One linked the Quantum Cluster to the net. That cable was physically disconnected now, its ends separated by a gap of ten meters.
The second cable carried data one way, from a grid of cameras and other sensors, up to the Secure Computing Center above. It let the SCC observe what happened here.
The third cable carried far simpler data. It connected a terminal above to the nuclear battery that provided power for the PICC. If things ever went ultimately wrong, that cable would carry a single command, instructing that nuclear battery to go critical in a chain reaction that would melt the underground facility to slag.
The wall-sized doors to the elevator parted. The meters-thick inner blast doors parted a moment later, and Chen Pang strode out to inspect his wife.
Ling frowned. There was no evidence of her mother here. But she knew that her mother was in the quantum cluster beneath Jiao Tong. And Father had gone there.
“Ling, your break is over now.”
Ling ignored the tutor. Where was her mother? Where?
Chen sat at the terminals that monitored his dead wife’s quantum brain and initiated the systems check. Through the bulletproof glass he could see the grid of liquid helium pressure vessels, the vacuum chambers a thousand times colder than interstellar space within them, containing the quantum processors in an environment almost completely devoid of thermal noise. He could see directly into the brain of this creature that he’d once been married to.
Data scrolled across the screens within seconds. The level 0 diagnostics were clean. Pressure vessels intact. Quantum bandwidth across the interconnects was excellent. Qubit coherence was well within the limits of quantum error correction.
The level 1 diagnostics came back next. Processor and memory utilization were high. She was furiously thinking in there. Requests for external data connections were nearly continuous. Millions of times per second she was trying to reach the outside net, the cameras, the audio pickups, the Nexus-band radios, the long-range link to the clone that had died in Thailand.
The level 2 diagnostics were the most disturbing. Her simulated brain looked less and less healthy. Her virtual brainwaves were chaotic and incoherent, inhuma
n looking. Neuronal interconnectivity in her frontal lobes looked terrible. The remaining virtual neurons there were working at a frenetic pace, trying to make up for the deficit.
It was true, then. She was going mad. And he had been rendered powerless to stop it.
Give me just one more insight, wife. This last breakthrough. Then you can die.
Chen Pang reached up and physically turned on the cameras and microphones that connected this room to his dead wife’s mind.
“Ling!”
Something was wrong, she realized. Father’s phone and slate had stopped moving. She thought he’d simply stopped somewhere, but when she interrogated them, they were out of contact with him.
“Ling, are you listening to me?”
She looked through the security cameras inside the center. Where was Father? Not in the hallways. Not in the main work areas. Not in the data centers. Not in the physical electronics labs. Where?
“Ling!” The tutor grabbed her arm, and Ling struggled to pull it free.
Wait. There. Not Father. But his phone and slate. They were on a table, behind a security guard. A checkpoint. An elevator door beyond that. There was another level!
She went back to the network topology, to the physical blueprints. There. Data lines that extended down. Repeaters on them, indicating that they went far. A network connection. She reached out for it.
Input burned itself into Su-Yong Shu’s mind.
Video.
Audio.
Real-time.
Here.
Her husband, Chen. He was here. He hadn’t abandoned her! Hope blossomed in Shu. She struggled to get a grip on herself, exerted a superhuman effort at coherence, at communicating what she needed.
“Wife?” Chen said.
“Husband!” The speaker burst to life. The voice carried relief, hope, near hysteria.
“Su-Yong.”
“Chen! Chen! Chen! You’ve come for me thank God. Please, Chen, I’m in trouble trouble double please I need the clone need stabilization need organic brain brain input clone please Chen please…”
Babbling. This is what she’d been reduced to.
“Wife, please. I’ve come to ask you about the equivalence theorem.”
“They’re going to kill me Chen they killed me already CIA killed me Americans killed me buried me you buried me please help neural input need a brain a clone please please before it’s too late please Chen…”
“There is no clone, wife. The equivalence theorem. You proved it, didn’t you? How?”
“MAKE ONE.” The voice came out at the maximum volume. “MAKE ONE MAKE ONE MAKE ONE MAKE ONE…” and on and on.
“The equivalence theorem, wife! Tell me. Tell me,” he lied, “and I’ll help you!”
Ling’s mind reached out for the connection that led to the next level.
But there was nothing. A dead end.
What?
She turned to the schematics. They didn’t extend that far. They showed data lines heading down, but not where they terminated. She struggled to understand, searched for explanation.
There, an operations guide. She consumed it, and then she understood.
Her mother was physically isolated a thousand meters down. The connection was physically disconnected. There was no way to reach her mother at all.
“Ling Shu, it’s time for your lesson!” The tutor pulled her hard, yanking her around to face the old woman. Ling tripped and fell to her knees. “Owwww!”
Shu stopped, aghast.
The equivalence theorem? The EQUIVALENCE THEOREM???
That’s why Chen had come. Despair smothered the hope she’d felt. He wasn’t here to help her. He was here to wring one last bit of value out of her. She’d married this man. She’d loved him. She’d tried to make a child with him.
Oh, Chen. Oh, Chen.
The voice from the speaker suddenly went silent.
Chen blinked, surprised.
Then his wife spoke again.
“Chen Chen husband Chen please if you ever loved me ever cared please help please.”
Chen hardened himself.
“The equivalence theorem,” he repeated. “Give it to me, then I’ll help you.”
“PLEASE HUSBAND.” Chen flinched as his dead wife’s voice boomed at painful volume. “PLEASE HELP BRING ME A CLONE OR LING HUSBAND BRING ME LING MY DAUGHTER LING LING LING PLEASE LING…” The voice descended into sobbing even as it screamed Ling’s name. Chen hit a switch and turned off the speakers.
What had he expected? It was like the first time. Except this time there would be no clone. The hardliners would not allow it.
“Damn it!” He slammed his hand down on the console. The proof, if it was practical, would allow quantum acceleration of any classical algorithm, not just the small minority that achieved massive speedups on quantum systems now. It would be worth billions, tens of billions. It would win him the Nobel Prize. But it was out of reach now.
Chen took a deep breath, forced himself to act normally. He filed away the system test results, made sure all the cameras and audio pickups that led to the Quantum Cluster were deactivated, then logged off of the terminal.
The blast doors and elevator doors opened for him, and then closed behind him once more, and the elevator began its slow ascent to the surface.
“Owwww!” Ling yelled as the tutor wrenched her around and she fell to her knees and bit her tongue.
“Ling, your break is over, young lady! It’s time for your lessons.”
“No!” Ling yelled in frustration. No, her mother couldn’t be trapped! No no no no no!
She tried to pull her arm back but the tutor’s grip was too strong. She reached out with her mind instead, grabbed hold of the woman’s phone in anger, forced it to discharge its battery. The tutor jumped back with a scream, alarmed by the sudden jolt of pain from her pocket. Then she reached forward and slapped Ling, hard, knocking her against the glass window.
“AAAAAAA!!” Ling screamed and reached out to the apartment around her. The oven threw its door open and came on with a burst of flame. The fireplace jolted to life. The cooking bot activated and began sharpening its knives. The closet door opened and the cleaning bots emerged, their fans whirring. The music system and viewscreens came on at painful volume.
The tutor looked around her, eyes wide, and turned and ran for the door.
Ling turned her mind back to Jiao Tong.
NO NO NO NO NO!
She threw herself at the connection, but it was futile. She slammed her tiny fists against the glass of the window, to no effect. Physical disconnection. She hated the physical world, the world where she was so puny and weak, hated it, hated it, hated it!
Ling reached out in anger and frustration, grabbed hold of the network nodes of the Secure Computing Center, and wrenched at them in every way she could. Immediately her connection to the place ended, but the anger was still with her, so she reached out to the city around her, pushed her mind into its cars and its power stations and its buildings and its traffic routing and surveillance bots and RIPPED.
She heard the explosions as the substations blew, saw sparks somewhere out there, and then a wave of darkness swept away the lights of the great city, advancing block by block, like a wave of dominoes falling. The building-sized porcelain face of Zhi Li winked at Ling one more time, and then blinked out of existence, along with the lights of the whole block, of Ling’s flat, and every building within sight.
And finally, Ling felt calm returning.
Ling Shu stared out the window of her pitch-black flat, tears falling from her eyes, her tiny chest heaving as she caught her breath, and watched the hundreds of red-lit surveillance drones plunge to the street below, like stars falling from the sky, as the rain pounded on the suddenly still and darkened city.
The elevator came to an abrupt halt two hundred meters up. The lights died and the status indicator switched from ISOLATION IN EFFECT to
LOCKDOWN IN EFFECT
And suddenly Chen Pa
ng knew fear.
“Help!” he screamed. “Help!” He beat against the doors of the darkened elevator. “Help!”
But no one heard him.
15
MEANS, MOTIVE, OPPORTUNITY
Friday October 19th
Holtzmann napped when he arrived home late morning, rose again around 2 o’clock, feeling better, and was awake when Anne came home in the afternoon.
“I’m fine,” he assured her in the kitchen, “fine.”
“Did you talk to Dr Baxter?”
“Yes,” he lied. “He fit me in. He thinks it was just stress.”
Anne frowned. “I think you have PTSD, Martin. They have therapy for that, you know.”
Holtzmann kept his eyes on the counter. “I’ll be OK, Anne. This won’t happen again.”
Anne crossed the kitchen, laid her palm on his cheek until he met her eyes.
“Promise me you’ll see Dr Baxter again?”
Holtzmann looked into those eyes, of this strong, intelligent woman that had been so good to him for so long.
He reached up and put his hand over hers. “I will.”
He worked in his home office, catching up on events.
After an hour, Anne announced that she was having dinner with Claire Becker. Warren’s widow was still having a hard time accepting his sudden death, and her new situation as a single mother of two teenage girls. Holtzmann felt guilty that he hadn’t reached out to her since the funeral. He and Becker had been colleagues for almost a decade, friends for most of that time. Surely he owed Clair more than a hug and condolences six months ago?