“But they were wrong. I was like a clock: I got wound up tighter and tighter, and in the end the spring just snapped and left me like this—broken and unmoving. All of that emotion I built up during my lifetime never got the chance to break free. Ah, it was wiped from the universe before it had a chance to be born.”
The FE fell still. The lift had reached its destination.
“Well,” Judy said, “I’m ready to meet you.”
The doors slid open.
“Hello, Judy,” said the Watcher.
The Watcher could take on any appearance that he chose. He habitually chose that of a young Japanese man, this time neatly dressed in a black passive suit.
“Hello,” Judy said. She giggled and turned around in the corridor, hugging herself tightly. “Well, what do we do now? What have you brought me here for?”
The Watcher was silent.
“Should we make love?” Judy laughed. “A symbolic union between yourself and humanity? That would make sense, wouldn’t it? And me a virgin, too. Keeping myself untouched and unspoilt for all these years.”
The Watcher seemed unperturbed. “I think you will calm down soon, Judy.”
“I think I will, yes. Or maybe I should cure you. Here I am, expert MTPH counselor, and you a broken mind. I could counsel you; put you back on the road to mental health. Is that what this is all about? Is that why you had me brought here?”
“No, Judy.”
“Then why? Why did you bring me here?”
The Watcher did not answer immediately. The corridor was silent. Just the sound of Judy and the Watcher breathing. The clean smell of the Watcher, containing the edge of something like cologne.
“Tell me this, Judy,” the Watcher said suddenly. “You’ve seen what it is like on Earth now. Every minute there is another infestation of Dark Seeds appearing somewhere on the planet.”
Viewing fields wobbled to life all around Judy and the Watcher. A Japanese garden of raked pebbles, dry grey rocks rising amongst them. Dark Seeds lay there among the regular patterns coaxed into the ground. In the background, colorful lines of people slowly walked away from the infection; clockwork rescue fliers were already dropping in from the sky.
“There are only enough fliers to save fifty percent of them,” the Watcher sighed at Judy’s side.
“Who shall we save?” He strode into the scene, his feet disturbing the elegantly raked stones of the garden. “Shall it be this young couple?” He pointed to two people who walked hand in hand away from the infection. “They will be so happy together. And yet, if I leave the woman behind, the two children following her will have a place on the flier.” The Watcher shook his head sadly. “A couple’s happiness or the promise of the future? Which should it be, Judy?”
He came back and stood directly before her.
“I make these decisions every day, Judy. What would you do?”
Judy grinned. “You pulled that trick on Eva Rye,” she said. “You fooled her into playing that game all those years ago. You fooled us all into playing along. Well, you don’t fool me any longer. The answer is: we don’t make the decisions. They make their own choices. The couple choose, the children choose, and they do it fairly. That’s what FE is all about, that’s what you should be all about, but your programming has got totally skewed. You’ve been wrong since the beginning. You haven’t been dealing with individuals; instead you’ve been trying to impose a perfect model on a group, trying to get them all to live in a certain identical way.”
The viewing fields shimmered and vanished. The Watcher was silent once more.
“I think you’ve been aware of that for some time,” Judy continued. “Now tell me, why am I here?”
The Watcher lost his impassive mask.
“You are fulfilling a Fair Exchange undertaken between Chris and myself, though I don’t think either of us realized how far-reaching the consequences would be. Come on, we need to go this way. Let’s see if you can interface your console with this building.”
Judy’s console plugged itself straight into the building’s datasphere. That was no surprise, as she was apparently DIANA property.
“This way,” said the Watcher, and he turned off the corridor and passed through a series of rooms ranged with low shapes, like half-submerged diamond whales. “Very high-capacity memory,” said the Watcher. “Normally they wouldn’t be this deep in a gravity well, they weigh so much, but DIANA must have wanted to keep their contents a secret.”
“You want to ask me what is in them, don’t you?”
“Each contains a human life,” said the Watcher, but he didn’t elaborate further.
The next room contained more of the massive shapes, and the next one. They passed room after room of semisubmerged diamond whales.
Finally, they passed into a different area of the complex and entered a low-ceilinged room containing a few sofas and a desk. A reception area.
“Through here,” said the Watcher. Beyond the reception area the whole feel of the building changed. It became more homey, more like a living area. They passed through another set of rooms, emerging finally into one that Judy recognized.
“A delivery room,” she said.
She looked around the familiar space and felt a sense of homecoming. She belonged here. The faint smell of talcum powder in the air brought a sense of smothering happiness to her.
There were thirteen cribs in the room, one for Judy and each of her twelve sisters. A sense array hung from the ceiling, just another shape amongst the glittering twirling mobiles that dangled down to entertain the newborns. The walls were decorated with bright primary-colored shapes that stimulated the mind and senses. The floor was something of an anticlimax, covered in a plain oatmeal carpet.
The Watcher spoke. “DIANA had a store of frozen embryos, brought from before the Transition, from before the time I took complete control of the running of human affairs. Thirteen aborted fetuses: as such they were not, legally speaking, human beings. DIANA brought them to term. DIANA regarded those thirteen babies as their property.”
“Oh?” said Judy. “What did they want with us?”
“They wanted to find out how you worked. Just how, exactly, your minds worked. DIANA had long been interested in intelligence. They wrote the AI known as Kevin, remember? They wanted to truly understand the nature of intelligence.”
“But surely they already understood? DIANA made digital personality constructs of humans back then. They were constructing AIs all the time.”
“No, Judy, I did. All of these things are the results of my technology. But DIANA was paranoid, maybe rightly so. They wanted to understand those principles for themselves. They constructed a program to examine the workings of the mind, and they incorporated it into the genetic structure of the children. You’re not saying anything now, Judy. You know that what I’m saying is the truth, don’t you?”
Judy felt the pressure of the fleshy cross on the back of her neck. She reached back and touched it.
“The meta-intelligence,” she whispered.
“Did you never think to look at yourself with it?” asked the Watcher. “That was what it was there for—”
“I don’t want to look at myself with it,” said Judy. “I don’t want to see that my mind is just a mechanical process. I don’t want to see that it’s just a Turing machine. Like the thing that runs this place.”
“So what? You say that as if there is something wrong with that.” The Watcher seemed indignant. “Your body is a mechanical process. Your heart pumps, your muscles contract, your nerves react. So what if your mind is a Turing machine? You are greater than the sum of your parts.”
Judy gave him a weak smile.
“I know that. But my eyes and ears and senses are just writing to a length of tape, and your words have just been written to that tape, and my brain is just the tape head that reads the words and then jumps back and forth as it reacts to what you said.” She couldn’t help herself now: she looked. A long reel of tape was threaded b
etween the hemispheres of her brain, clicking through a section at a time, chattering back and forth as she examined his face, eyes darting.
“No,” said Judy, turning the gaze of the meta-intelligence away from herself. “I know you’re humoring me,” she said. “I know that you are. I don’t blame you. I know that a Turing machine is just a mathematical concept. But, I look through this and I can feel my brain mapping directly onto the mechanism. It’s like I can almost see the original process in there, just out of reach: the self-referential part of my mind that allows me to be me. And if I see that, I will have defined myself and all of my thoughts.”
“And now look away,” said the Watcher. “Look away, Judy. Don’t look back again.”
She did as she was told. She wanted to do as she was told.
The Watcher went on. “Do you see the danger, Judy? I think you do now. The meta-intelligence program was a good idea, but it was observed by other AIs. AIs within DIANA and, later on, outside of DIANA. The algorithm behind the program became an idea that took root in AIs’ minds, and then it was passed on to humans, imperfectly understood. A human could almost look into their own mind and become transfixed by the sight of the mechanism. This is how the White Death was born.”
“The White Death,” said Judy, reeling with the revelation. She had experienced the effect before, secondhand. But now she understood. Now she understood the spiral that drew the mind in upon itself until it was thinking about nothing more or less than its own processing. Trapped in Recursion.
“The White Death,” she repeated. “I understand now.” Her voice hardened. “So where do you come into all this?”
“Right here,” said the Watcher. A scene sprang to life on Judy’s console. “This is stored in the building’s surveillance net. October the twenty-sixth, 2211.”
Viewing fields wobbled into life in the delivery room; they quickly took on the appearance of the room itself. Nothing had changed save for the fact that thirteen babies now lay in the cots. Three months old, Judy guessed. They looked at the mobiles with bright blue eyes, drew their legs up to their tummies, yawned and rubbed their eyes with little fists, opened little pink mouths to cry, and waited for the nurses to come to them with their smart pinstriped aprons.
One young man stood over a cot, holding his hand over the baby’s face.
“You make me laugh when you do that, Henry,” said an older woman, as she lifted the happy pink child out of its cot.
“I’m not doing anything, Margaret.” Henry snatched his hand away. The baby in the cot was sleeping peacefully, its little fists on the pillow on either side of its head.
That’s me, thought Judy. That’s me Henry was looking at.
Margaret bounced the baby expertly on her shoulder, one arm wrapped around its little bottom, the other pointing upwards.
“She’s only sleeping,” she said. “The sense cluster would pick it up if she wasn’t breathing. There’s no need for you to keep feeling for her breath.”
Judy gulped as the man picked up her younger self. He had such a kind face, she thought. His light brown hair was already receding, his chin a little too long, but when he placed the baby on his shoulder and rocked it gently in its sleep, a look of such warmth came over his face. Judy was a ghost in the recorded scene; she moved close to him and a lump rose to her throat as she watched him tilt his head around at an awkward angle in order to get a better look at the baby’s face. She saw the way he surreptitiously licked a finger and raised it to just underneath the baby’s nose, the better to catch its slightest breath. And her eyes welled with tears as she caught the contented smile as he found what he was looking for. Just how many times, she wondered, had she lain in this cot and half woken from a dream in which her forming mind twisted over itself to get a better look at its developing consciousness? What sort of nightmares must she have experienced in that recursive, self-referential world? And then to have opened her eyes and to have seen a hand just above her face, reaching down on the end of an impossibly long arm.
She started to cry, tears bubbling up and streaking her cheeks. She wiped them away, and smiled through bleary eyes.
All that time and she had never realized. Every time she was anxious, she had experienced that dream. It wasn’t a bad thing at all. It was her subconscious reminding her that she had once been loved.
She now followed Henry around the room, watched him bouncing her infant self on his shoulder, watched him feeding her from a bottle, watched him help her sit up amongst the other babies on the gaily colored mat that had been rolled out across the floor.
When the Watcher spoke again, the sound of his voice made her jump.
“Of course there was someone else here,” he said, “someone who the surveillance systems could not pick up. I’ll fill in the gaps.”
It seemed like the grey crystalline robot had been standing in the corner all along and Judy had just registered his presence. He stood, arms patiently folded, looking around the room with amused patience on his beautiful face.
“Chris,” said Judy. “I should have guessed. What was he doing here?”
“Monitoring the room for me,” said the Watcher. “I was going to perform a little experiment of my own.”
“It was performed on me, wasn’t it?”
“Yes, Judy. You were born in 2211, the same year that I performed another experiment to try to determine the truth of my own origins. I placed a developing mind in the ziggurat under the stars on a distant planet, to see if it would become infected by the virus that made me. Do you think that was the only test that I made? Minds can live in many containers, in machinery and in flesh. The human mind is just an AI that has evolved within a set of grey cells.”
Judy’s eyes widened, guessing what the Watcher was going to say next.
“I wiped the minds of those thirteen babies. Left them empty, waiting to see if anything would develop there.”
Judy felt as if she had been stabbed in the stomach. She felt the knife in there, twisting, tearing her life apart.
“You did that to me?” she whispered.
“No,” said the Watcher, “I did it to the baby that Henry there now holds in his arms. You are not that baby. You are what developed afterwards.”
Judy couldn’t speak, the moment was too big. She held her stomach, she bit her lip, then she rubbed her dry eyes. She needed to think. The Watcher, however, would not be quiet.
“Chris once told you that you would come around to his point of view someday: that you would want to help him to destroy me.”
“I never believed him, until a moment ago,” said Judy. “Now I see it’s true. He was right.”
“So what are you going to do?”
“Nothing,” Judy said. “You’re already defeated. The Dark Seeds are all over this planet, and you can no longer fight them. All you can do is hold on to what little power you have remaining. Sooner or later you’re going to have to climb into a sealed processing space and stay there.”
“What about all the people outside? All the people who live on Earth? Don’t you care for them?”
“Yes, of course I do,” Judy snapped. “I was a Social Care operative for years. But it’s funny; Eva Rye got into my mind, and I saw things through her eyes. I saw the way you manipulated her to get what you wanted. I had a friend once called Frances. She was an AI. Someone said that she used my personality as a template for her own. It was a negative template, but a template nonetheless. I’m beginning to think that you did the same with Eva…”
“I don’t deny it.”
“…and that makes me wonder. I think about Kevin—you know he claims that he is not an AI. He says he passes the Turing test every day, but he is not intelligent: just a sequence of yes/no responses, just a massive algorithm.”
“Yes?”
“And I wonder, are you any different? Are you an AI, or are you just a reflection of all of us? You appear to have feelings, but all we are seeing in your actions are our own emotions reflecting right back at
us. Wouldn’t that be the ultimate irony? The last two hundred years of history have been shaped by you, yet you’re not even intelligent. We just took you at your word when you said you were.”
“It’s a clever theory, Judy,” the Watcher looked smug, “but what about all the other personality constructs? What about your sisters? Maybe they weren’t intelligent. Maybe they too were just reflections of human emotions.”
Judy hugged herself. Was this the ultimate betrayal? Was she denying those digital copies of herself their supposed existence? And yet she had to go on.
“I don’t know. Were they intelligent, or did they just think they were?”
The Watcher laughed.
“You should know. You used to go into the digital world. You have spoken with personality constructs in there! You know they were intelligent.”
“Or was I just seeing myself reflected back again?”
In the virtual scene, Henry placed the baby that would become Judy back in its crib. At that, Judy turned her back on the Watcher and made her way across the room. She bent closer and regarded the look of tender satisfaction on Henry’s young face, watched the way he pulled the pink blanket up over the sleeping child’s chest. She smiled as he gently pressed the baby’s little nose.
“Beep,” he whispered under his breath, then he turned and walked from the room, this whole scene observed by the silent robot in the corner.
“And how do you know that you have intelligence, Judy?” asked the Watcher. “Or do you just think that you have?”
“I’m the only one who can tell the difference.”
“Good answer,” the Watcher said. “But you could also discern whether the rest of us do, if you only took the trouble to think about it—even without the use of your meta-intelligence. You can tell that I have intelligence, because I can see the Dark Seeds and fix them in position.” He looked thoughtful. “That’s what it takes to be intelligent,” the Watcher continued, “the ability to observe. Now, that’s enough of this. Come on, we’re almost done here.”
Divergence Page 30