“That’s right.”
“Then we want the same thing. What I want is for you to succeed.”
She ran a hand over her face; whatever else she was thinking, there was no denying that she was tempted. “It’s gratifying that you have so much confidence in my abilities,” she said. “But we need to be clear about some things. This prototype is amazing, and if you ever get the production costs down I’m sure it will have some extraordinary applications. It would eat up climate forecasting, lattice QCD, astrophysical modelling, proteomics . . .”
“Of course.” Actually, Daniel had no intention of marketing the device. He’d bought out the inventor of the fabrication process with his own private funds; there were no other shareholders or directors to dictate his use of the technology.
“But AI,” Julie said, “is different. We’re in a maze, not a highway; there’s nowhere that speed alone can take us. However many exaflops I have to play with, they won’t spontaneously combust into consciousness. I’m not being held back by the university’s computers; I have access to SHARCNET anytime I need it. I’m being held back by my own lack of insight into the problems I’m addressing.”
Daniel said, “A maze is not a dead end. When I was twelve, I wrote a program for solving mazes.”
“And I’m sure it worked well,” Julie replied, “for small, two-dimensional ones. But you know how those kind of algorithms scale. Put your old program on this crystal, and I could still design a maze in half a day that would bring it to its knees.”
“Of course,” Daniel conceded. “Which is precisely why I’m interested in hiring you. You know a great deal more about the maze of AI than I do; any strategy you developed would be vastly superior to a blind search.”
“I’m not saying that I’m merely groping in the dark,” she said. “If it was that bleak, I’d be working on a different problem entirely. But I don’t see what difference this processor would make.”
“What created the only example of consciousness we know of?” Daniel asked.
“Evolution.”
“Exactly. But I don’t want to wait three billion years, so I need to make the selection process a great deal more refined, and the sources of variation more targeted.”
Julie digested this. “You want to try to evolve true AI? Conscious, human-level AI?”
“Yes.” Daniel saw her mouth tightening, saw her struggling to measure her words before speaking.
“With respect,” she said, “I don’t think you’ve thought that through.”
“On the contrary,” Daniel assured her. “I’ve been planning this for twenty years.”
“Evolution,” she said, “is about failure and death. Do you have any idea how many sentient creatures lived and died along the way to Homo sapiens? How much suffering was involved?”
“Part of your job would be to minimize the suffering.”
“Minimize it?” She seemed genuinely shocked, as if this proposal was even worse than blithely assuming that the process would raise no ethical concerns. “What right do we have to inflict it at all?”
Daniel said, “You’re grateful to exist, aren’t you? Notwithstanding the tribulations of your ancestors.”
“I’m grateful to exist,” she agreed, “but in the human case the suffering wasn’t deliberately inflicted by anyone, and nor was there any alternative way we could have come into existence. If there really had been a just creator, I don’t doubt that he would have followed Genesis literally; he sure as hell would not have used evolution.”
“Just, and omnipotent,” Daniel suggested. “Sadly, that second trait’s even rarer than the first.”
“I don’t think it’s going to take omnipotence to create something in our own image,” she said. “Just a little more patience and self-knowledge.”
“This won’t be like natural selection,” Daniel insisted. “Not that blind, not that cruel, not that wasteful. You’d be free to intervene as much as you wished, to take whatever palliative measures you felt appropriate.”
“Palliative measures?” Julie met his gaze, and he saw her expression flicker from disbelief to something darker. She stood up and glanced at her wristphone. “I don’t have any signal here. Would you mind calling me a taxi?”
Daniel said, “Please, hear me out. Give me ten more minutes, then the helicopter will take you to the airport.”
“I’d prefer to make my own way home.” She gave Daniel a look that made it clear that this was not negotiable.
He called her a taxi, and they walked to the elevator.
“I know you find this morally challenging,” he said, “and I respect that. I wouldn’t dream of hiring someone who thought these were trivial issues. But if I don’t do this, someone else will. Someone with far worse intentions than mine.”
“Really?” Her tone was openly sarcastic now. “So how, exactly, does the mere existence of your project stop this hypothetical bin Laden of AI from carrying out his own?”
Daniel was disappointed; he’d expected her at least to understand what was at stake. He said, “This is a race to decide between Godhood and enslavement. Whoever succeeds first will be unstoppable. I’m not going to be anyone’s slave.”
Julie stepped into the elevator; he followed her.
She said, “You know what they say the modern version of Pascal’s Wager is? Sucking up to as many Transhumanists as possible, just in case one of them turns into God. Perhaps your motto should be ‘Treat every chatterbot kindly, it might turn out to be the deity’s uncle.’”
“We will be as kind as possible,” Daniel said. “And don’t forget, we can determine the nature of these beings. They will be happy to be alive, and grateful to their creator. We can select for those traits.”
Julie said, “So you’re aiming for übermenschen that wag their tails when you scratch them behind the ears? You might find there’s a bit of a trade-off there.”
The elevator reached the lobby. Daniel said, “Think about this, don’t rush to a decision. You can call me any time.” There was no commercial flight back to Toronto tonight; she’d be stuck in a hotel, paying money she could ill-afford, thinking about the kind of salary she could demand from him now that she’d played hard to get. If she mentally recast all this obstinate moralizing as a deliberate bargaining strategy, she’d have no trouble swallowing her pride.
Julie offered her hand, and he shook it. She said, “Thank you for dinner.”
The taxi was waiting. He walked with her across the lobby. “If you want to see AI in your lifetime,” he said, “this is the only way it’s going to happen.”
She turned to face him. “Maybe that’s true. We’ll see. But better to spend a thousand years and get it right, than a decade and succeed by your methods.”
As Daniel watched the taxi drive away into the fog, he forced himself to accept the reality: she was never going to change her mind. Julie Dehghani had been his first choice, his ideal collaborator. He couldn’t pretend that this wasn’t a setback.
Still, no one was irreplaceable. However much it would have delighted him to have won her over, there were many more names on his list.
2
Daniel’s wrist tingled as the message came through. He glanced down and saw the word PROGRESS! hovering in front of his watch face.
The board meeting was almost over; he disciplined himself and kept his attention focused for ten more minutes. WiddulHands.com had made him his first billion, and it was still the pre-eminent social networking site for the 0–3 age group. It had been fifteen years since he’d founded the company, and he had since diversified in many directions, but he had no intention of taking his hands off the levers.
When the meeting finished he blanked the wall screen and paced the empty conference room for half a minute, rolling his neck and stretching his shoulders. Then he said, “Lucien.”
Lucien Crace appeared on the screen. “Significant progress?” Daniel enquired.
“Absolutely.” Lucien was trying to maintain polite eye
contact with Daniel, but something kept drawing his gaze away. Without waiting for an explanation, Daniel gestured at the screen and had it show him exactly what Lucien was seeing.
A barren, rocky landscape stretched to the horizon. Scattered across the rocks were dozens of crab-like creatures – some deep blue, some coral pink, though these weren’t colours the locals would see, just species markers added to the view to make it easier to interpret. As Daniel watched, fat droplets of corrosive rain drizzled down from a passing cloud. This had to be the bleakest environment in all of Sapphire.
Lucien was still visible in an inset. “See the blue ones over by the crater lake?” he said. He sketched a circle on the image to guide Daniel’s attention.
“Yeah.” Five blues were clustered around a lone pink; Daniel gestured and the view zoomed in on them. The blues had opened up their prisoner’s body, but it wasn’t dead; Daniel was sure of that, because the pinks had recently acquired a trait that turned their bodies to mush the instant they expired.
“They’ve found a way to study it,” Lucien said. “To keep it alive and study it.”
From the very start of the project, he and Daniel had decided to grant the Phites the power to observe and manipulate their own bodies as much as possible. In the DNA world, the inner workings of anatomy and heredity had only become accessible once highly sophisticated technology had been invented. In Sapphire, the barriers were designed to be far lower. The basic units of biology here were “beads”, small spheres that possessed a handful of simple properties but no complex internal biochemistry. Beads were larger than the cells of the DNA world, and Sapphire’s diffractionless optics rendered them visible to the right kind of naked eye. Animals acquired beads from their diet, while in plants they replicated in the presence of sunlight, but unlike cells they did not themselves mutate. The beads in a Phite’s body could be rearranged with a minimum of fuss, enabling a kind of self-modification that no human surgeon or prosthetics engineer could rival – and this skill was actually essential for at least one stage in every Phite’s life: reproduction involved two Phites pooling their spare beads and then collaborating to “sculpt” them into an infant, in part by directly copying each other’s current body plans.
Of course these crabs knew nothing of the abstract principles of engineering and design, but the benefits of trial and error, of self-experimentation and cross-species plagiarism, had led them into an escalating war of innovation. The pinks had been the first to stop their corpses from being plundered for secrets, by stumbling on a way to make them literally fall apart in extremis; now it seemed the blues had found a way around that, and were indulging in a spot of vivisection-as-industrial-espionage.
Daniel felt a visceral twinge of sympathy for the struggling pink, but he brushed it aside. Not only did he doubt that the Phites were any more conscious than ordinary crabs, they certainly had a radically different relationship to bodily integrity. The pink was resisting because its dissectors were of a different species; if they had been its cousins it might not have put up any fight at all. When something happened in spite of your wishes, that was unpleasant by definition, but it would be absurd to imagine that the pink was in the kind of agony that an antelope being flayed by jackals would feel – let alone experiencing the existential terrors of a human trapped and mutilated by a hostile tribe.
“This is going to give them a tremendous advantage,” Lucien enthused.
“The blues?”
Lucien shook his head. “Not blues over pinks; Phites over tradlife. Bacteria can swap genes, but this kind of active mimetics is unprecedented without cultural support. Da Vinci might have watched the birds in flight and sketched his gliders, but no lemur ever dissected the body of an eagle and then stole its tricks. They’re going to have innate skills as powerful as whole strands of human technology. All this before they even have language.”
“Hmm.” Daniel wanted to be optimistic too, but he was growing wary of Lucien’s hype. Lucien had a doctorate in genetic programming, but he’d made his name with FoodExcuses.com, a web service that trawled the medical literature to cobble together quasi-scientific justifications for indulging in your favourite culinary vice. He had the kind of technobabble that could bleed money out of venture capitalists down pat, and though Daniel admired that skill in its proper place, he expected a higher insight-to-bullshit ratio now that Lucien was on his payroll.
The blues were backing away from their captive. As Daniel watched, the pink sealed up its wounds and scuttled off towards a group of its own kind. The blues had now seen the detailed anatomy of the respiratory system that had been giving the pinks an advantage in the thin air of this high plateau. A few of the blues would try it out, and if it worked for them, the whole tribe would copy it.
“So what do you think?” Lucien asked.
“Select them,” Daniel said.
“Just the blues?”
“No, both of them.” The blues alone might have diverged into competing subspecies eventually, but bringing their old rivals along for the ride would help to keep them sharp.
“Done,” Lucien replied. In an instant, ten million Phites were erased, leaving the few thousand blues and pinks from these badlands to inherit the planet. Daniel felt no compunction; the extinction events he decreed were surely the most painless in history.
Now that the world no longer required human scrutiny, Lucien unthrottled the crystal and let the simulation race ahead; automated tools would let them know when the next interesting development arose. Daniel watched the population figures rising as his chosen species spread out and recolonized Sapphire.
Would their distant descendants rage against him, for this act of “genocide” that had made room for them to flourish and prosper? That seemed unlikely. In any case, what choice did he have? He couldn’t start manufacturing new crystals for every useless side-branch of the evolutionary tree. Nobody was wealthy enough to indulge in an exponentially growing number of virtual animal shelters, at half a billion dollars apiece.
He was a just creator, but he was not omnipotent. His careful pruning was the only way.
3
In the months that followed, progress came in fits and starts. Several times, Daniel found himself rewinding history, reversing his decisions and trying a new path. Keeping every Phite variant alive was impractical, but he did retain enough information to resurrect lost species at will.
The maze of AI was still a maze, but the speed of the crystal served them well. Barely eighteen months after the start of Project Sapphire, the Phites were exhibiting a basic theory of mind: their actions showed that they could deduce what others knew about the world, as distinct from what they knew themselves. Other AI researchers had spliced this kind of thing into their programs by hand, but Daniel was convinced that his version was better integrated, more robust. Human-crafted software was brittle and inflexible; his Phites had been forged in the heat of change.
Daniel kept a close watch on his competitors, but nothing he saw gave him reason to doubt his approach. Sunil Gupta was raking in the cash from a search engine that could “understand” all forms of text, audio and video, making use of fuzzy logic techniques that were at least forty years old. Daniel respected Gupta’s business acumen, but in the unlikely event that his software ever became conscious, the sheer cruelty of having forced it to wade through the endless tides of blogorrhoea would surely see it turn on its creator and exact a revenge that made The Terminator look like a picnic. Angela Lindstrom was having some success with her cheesy AfterLife, in which dying clients gave heart-to-heart interviews to software that then constructed avatars able to converse with surviving relatives. And Julie Dehghani was still frittering away her talent, writing software for robots that played with coloured blocks side-by-side with human infants, and learnt languages from adult volunteers by imitating the interactions of baby talk. Her prophesy of taking a thousand years to “get it right” seemed to be on target.
As the second year of the project drew to a c
lose, Lucien was contacting Daniel once or twice a month to announce a new breakthrough. By constructing environments that imposed suitable selection pressures, Lucien had generated a succession of new species that used simple tools, crafted crude shelters, and even domesticated plants. They were still shaped more or less like crabs, but they were at least as intelligent as chimpanzees.
The Phites worked together by observation and imitation, guiding and reprimanding each other with a limited repertoire of gestures and cries, but as yet they lacked anything that could truly be called a language. Daniel grew impatient; to move beyond a handful of specialized skills, his creatures needed the power to map any object, any action, any prospect they might encounter in the world into their speech, and into their thoughts.
Daniel summoned Lucien and they sought a way forward. It was easy to tweak the Phites’ anatomy to grant them the ability to generate more subtle vocalizations, but that alone was no more useful than handing a chimp a conductor’s baton. What was needed was a way to make sophisticated planning and communications skills a matter of survival.
Eventually, he and Lucien settled on a series of environmental modifications, providing opportunities for the creatures to rise to the occasion. Most of these scenarios began with famine. Lucien blighted the main food crops, then offered a palpable reward for progress by dangling some tempting new fruit from a branch that was just out of reach. Sometimes that metaphor could almost be taken literally: he’d introduce a plant with a complex life cycle that required tricky processing to render it edible, or a new prey animal that was clever and vicious, but nutritionally well worth hunting in the end.
Time and again, the Phites failed the test, with localized species dwindling to extinction. Daniel watched in dismay; he had not grown sentimental, but he’d always boasted to himself that he’d set his standards higher than the extravagant cruelties of nature. He contemplated tweaking the creatures’ physiology so that starvation brought a swifter, more merciful demise, but Lucien pointed out that he’d be slashing his chances of success if he curtailed this period of intense motivation. Each time a group died out, a fresh batch of mutated cousins rose from the dust to take their place; without that intervention, Sapphire would have been a wilderness within a few real-time days.
The Mammoth Book of Best New Science Fiction 22nd Annual Collection Page 48