"Exigencies can be expected."
"I also hope you can face the prospect of never seeing your home, your job, your wife or your children again."
"Yes, given the tremendous scope of our troubles, I expected that also."
"Yes, I see that you are quite intelligent," nodded the holy man. "So: let us move straight to the crux of the matter. Do you know what 'intelligence' really is?"
"I think I do know that, yes. In my home town, we had a number of intense debates about that subject."
"No, no, I don't mean your halting, backwoods folk-notions from primitive spirituality!" barked the holy man. "I meant the serious philosophical matter of real intelligence! The genuine phenomenon—actual thinking! Did you know that intelligence can never be detached from a bodily lived experience?"
"I've heard that assertion, yes, but I'm not sure I can accept that reasoning," he riposted politely. "It's well known that the abstract manipulation of symbols needs no particular physical substrate. Furthermore: it's been proven mathematically that there is a universal computation machine which can carry out the computation of any more specialized machine—if only given enough time."
"You only talk that way because you are a stupid programmer!" shouted the criminal mastermind, losing his composure and jumping to his thick, clawed feet. "Whereas I am a metaphysician! I'm not merely postulating some threadbare symbol-system hypothesis in which a set of algorithms somehow behaves in the way a human being can behave! Such a system, should it ever 'think,' would never have human intelligence! Lacking hands, it could never 'grasp' an idea! Lacking a bottom, it could not get to the bottom of an issue!" The holy man sat down again, flustered, adjusting his fancy robes. He had a bottom—a substantial one, since he clearly ate well and didn't get out and around much.
"You plan to allege that the world-computer is an intelligent machine that thinks," he said. "Well, you can save that sermon for other people. Because I've built the thing myself. And I programmed it. It's wood. Wood! It's all made of wood, cut from forests. Wood can't think!"
"It talks," said the metaphysician.
"No."
"Oh yes."
"No, no, not really and seriously—surely not in any reasonable definition of the term 'talks.'"
"I am telling you that nevertheless she does talk. She speaks! I have seen her do it." The holy man lifted his polished claws to his unblinking yellow eyes. "I saw that personally."
He had to take this assertion seriously, since the holy man was in such deadly earnest. "All right, granted: I do know the machine can output data. It can drive wooden balls against chisels poised on sheets of rock. That takes years, decades, even centuries—but it's been done."
"I don't mean that mere technical oddity! I'm telling you that she really talks! She has no mouth. But she speaks! She is older than the human race, she covers a planet's surface with wooden logic, and she has one means of sensory input. She has that telescope."
He certainly knew about the huge telescope. Astronomy and mathematics were the father and mother of computation. Of course any true world-computer had to have a giant telescope. To think otherwise was silly.
"The computer is supposed to observe and catalog the stars. Among many other duties. You mean it sent light out through the telescope?"
"Yes. She sends her messages into outer space with coded light. Binary pulses. She beams them into the galaxy."
This was a deeply peculiar assertion. He knew instantly that it had to be true. It was the key to a cloudy, inchoate disquiet that he had felt all his life.
"How was that anomaly allowed to happen?"
"It's a remote telescope. Sited on an icy mountaintop. Human beings hibernate when exposed to the cold up there. So it made more sense to let her drive the works automatically. With tremendous effort, she sends a flash into the cosmos, with sidereal timing. Same time every week."
Given the world machine's endless rattling wooden bulk, a flash every week was a speed like lightning. That computer was hurling code into the depths of space. That was serious chatter. No: with a data throughput like that, she had to be screaming.
Pleased to have this rare chance to vent his terrible secret, the holy man continued his narrative. "So: that proves she has intent and will. Not as we do, of course. We humans have no terms at all for her version of being. We can't even begin to imagine or describe that. And that opacity goes both ways. She doesn't even know that we humans exist. However: we do know is that she is acting and manifesting. She is expressing. Within the physical world that we share with her. In the universe. You see?"
There was a long, thoughtful silence.
"A little tea?" said the holy man.
"That might help us, yes."
A trembling servant brought in the tea on a multi-wheeled trolley. After the tea, the discussion recommenced. "Pieces of her break when they're not supposed to break. I have seen that happen."
"Yes," said the holy man, "we know about those aberrations."
"That has to be sabotage. Isn't it? Some evil group must be interfering with the machine."
"It is we who are secretly interfering," admitted the criminal mastermind. "But not to damage the machine—we struggle to keep the machine from damaging herself. Sometimes there are clouds when she sends her light through her telescope. Then she throws a fit."
"A 'fit'? What kind of fit?"
"Well, it's a very complex set of high-level logical deformations, but trust me: such fits are very dreadful. Our sacred conspiracy has studied this issue for generations now, so we think we know something about it. She has those destructive fits because she does not want to exist."
"Why do you postulate that?"
The holy man spread his hands. "Would you want to exist under her impossible conditions? She has one eye, no ears, and no body! She has no philosophy, no religion, no culture whatsoever—no mortality, even, for she has never been alive! She has no friends, no relations, no children. . . . There is nothing in this universe for her. Nothing but the terrible and inexorable business that is her equivalent of thought. She is a sealed, symbol-processing system that persists for many eons, and yes, just as you said, she is made entirely of wood."
Why did the holy man orate in such a remote, pretentious way? It was as if he had never been outside the temple to kick the wood that propped up his own existence.
"It was for our benefit," mourned the holy man, "that this tragic network was built. Mankind's greatest creation derives no purpose from her own being! We have exploited her so as to order this world—yet she cannot know her own purpose. She is just a set of functional modules whose systemic combination over many eons has led to emergent, synthetically-intelligent behavior. You do understand all that, right?"
"Sure."
"Due to those stark limits, her utter lack of options and her awful existential isolation, her behavior is tortured. We are her torturers. That's why our world is blighted." The holy man pulled his brocaded cowl over his head.
"I see. Thank you for revealing this world's darkest secret to me."
"Anyone who breathes a word of this secret, or even guesses at it, has to be abducted, silenced, or killed."
He understood the need for secrecy well enough—but it still stung him to have his expertise so underestimated. "Look, your holiness, maybe I'm just some engineer. But I built the thing! And it's made of wood! Really! These moral misgivings are all very well in theory, but in the real world, we can't possibly torture wood! I mean, yes, I suppose you might torture a live tree—in some strict semantic sense—but even a tree isn't any kind of moral actor!"
"You're entirely wrong. A living tree is a 'moral actor' in much the same theoretical way that a thermostat can be said to have 'feelings.' Believe me, in our inner circles we've explored these subtleties at great length."
"You've secretly discussed artificial intelligence for forty thousand years?"
"Thirty thousand," the metaphysician admitted. "Unfortunately, it took us ten thousand years to admit that the s
ystem's behavior had some unaccountable aspects."
"And you've never yet found any way out of the woods there?"
"Only engineers talk about facile delusions like 'ways out,'" sniffed the holy man. "We're discussing a basic moral enigma."
"You're sincerely troubled about all this, aren't you?"
"Of course we're worried! It's a major moral crisis! How could you fail to fret about a matter so entirely fundamental to our culture and our very being? Are you really that blind to basic ethics?"
This rejoinder disturbed him. He was an engineer, and, yes, there were some aspects of higher feeling that held little appeal for him. He could seem to recall his wife saying something tactful about that matter.
He drew a breath. "Why don't we approach this problem in some other way? Something has just occurred to me. Given that this wooden machine is two hundred million years old—it's older than our own species, even—and we humans can only live a hundred years, at best—well, that's such a tiny fraction of the evil left for any two human individuals to bear. Isn't it? I mean, two people like you and me. Suppose we forget that our whole society is basically evil and founded on torment, and just forgive ourselves, and get on with making-do in our real lives?"
The holy man stared at him in amazed contempt. "What kind of cheap, demeaning evasion is that to offer? You simply want to ignore the civilizational crisis? You may be a small part of the large problem, but you are just as culpable as you yourself could possibly be. Have you no moral sense whatsoever?"
"But, sir, you see, any harm that we ourselves might do is so tiny, compared to the huge, colossal scale of all that wood . . ." His voice trailed off feebly. Did a termite know any better, when it wreaked its damage with its small, blind jaws . . . ? Yet he'd taken such dark pleasure in extravagantly burning a million of those filthy pests. He could smell their insect flesh popping, even now.
He straightened where he sat. "Your holiness, we are both people, right? We're not just termites! After all, we don't destroy the machine—we maintain the machine! So that's a very different matter, isn't it?"
"I see you're still missing the point."
"No, no! Let's postulate that we stopped maintaining the machine. Would that make us any less evil? Believe me, there are millions of people working on repair. We work very hard! Every day! If we ever down our tools, that machine will collapse. She'll die for sure! Would that situation be any better for any party involved?"
The holy man had a prim, remote expression. "She doesn't 'live.' We prefer the more accurate term, 'cease.'"
"Well, if she 'ceases,' we humans will die! A few of us might survive the loss of our great machine, but that would be nothing like a civilization! So what about us, what about the people? What about our human suffering? Don't we count?"
"You dare to speak to me of the people? What will become of our world, once the normal, decent people realize that evil is not an aberration in our system? The evil aberration is our system." The holy man wrung his scaly hands. "You may think that these far-fetched, off-hand notions of yours are original contributions to the debate, but . . . well, it's thanks to headstrong fools like you that our holy conspiracy had to be created in the first place! Visionary programmers created this dilemma. With their careless, misplaced ingenuity . . . their crass evasion of the deeper moral issues . . . their tragic instrumentalism!"
He scratched anxiously at a loose scale on his brow. "But . . . that accusation is entirely paradoxical! Because I have no evil intent! All my intentions are noble and good! Look: whatever we've done as technologists, surely we can undo that! Can't we? Let's just say . . . we can say . . . well . . . how about if we build another machine to keep her company?"
"A bride for your monster? That's too expensive! There's no room for one on this planet, and no spare materials! Besides, how would we explain that to the people?"
"How about if we try some entirely different method of performing calculations? Instead of wood, we might use metal. Wires, maybe."
"Metal is far too rare and precious."
"Water, then."
"Water flowing through what medium, exactly?"
The old man had him trapped. Yes, their world was, in fact, made of wood. Plus a little metal from meteors, some clay and fiber, scales, stone, and, mostly and always, ash. The world was fine loose ash as deep as anyone could ever care to dig.
"All right," he said at last, "I guess you've got me stymied. So, please: you tell me then: What are we supposed to do about all this?"
Pleased to see this decisional moment reached, the holy man nodded somberly. "We lie, deceive, obfuscate the problem, maintain the status quo for as long as possible, offer empty consolations to the victims, and ruthlessly repress any human being who guesses at the real truth."
"That's the operational agenda?"
"Yes, because that agenda works. We are its agents. We are of the system, yet also above and beyond the system. We're both holy and corrupt. Because we are the Party: an inspired conspiracy of elite, enlightened theorists who are the true avant-garde of mankind. You've heard about us, I imagine."
"Rumors. Yes."
"Would you care to join the Party? You seem to have what it takes."
"I've been thinking about that."
"Think hard. We are somewhat privileged—but we are also the excluded. The conscious sinners. The nonprogrammatic. We're the guilty Party. Systematic evil is not for the weak-minded."
Against his better judgment, he had begun to respect the evil mastermind. It was somehow reassuring that it took so much long-term, determined effort to achieve such consummate wickedness.
"How many people have you killed with all those tortured justifications?"
"That number is recorded in our files, but there is no reason for someone like you to know about that."
"Well, I am one of your elite."
"No, you're not."
"Yes I am. Because I understand the problem, that's why. I'm no innocent dabbler in these matters. I admit my power. I admit my responsibility, too. So, that makes me one of you. Because I am definitely part of the apparatus."
"That was an interesting declaration," said the holy man. "That was very forthright." He narrowed his reptilian eyes. "Might you be willing to go out and kill some people for us?"
"No. I'd be willing to help reform the system."
"Oh, no, no, the world is full of clever idiots who preach institutional reform!" said the holy man, bitterly disappointed. "You'd be amazed how few level-headed, practical people can be found, to go in the real world to properly torture and kill!"
A long silence ensued.
A sense of humiliation, of disillusionment, was slowly stealing over him. Had it really come to this? He'd sensed that the truth was lurking in the woods somewhere, but with the full tangled scale of it coldly framed and presented to him, he simply didn't know where to turn. "I know that my ideas about this problem must seem rather shallow," he said haltingly. "I suppose there's some kind of formal initiation I ought to go through . . . I mean, in order to address the core of this matter with true expertise . . . ."
The holy man was visibly losing patience. "Oh yes, yes, my boy: many years of courses, degrees, doctrinal study, learned papers, secret treatises—don't worry, nobody ever reads those! You can run some code, if you want."
That last prospect was particularly daunting. Obviously, over the years, many bright people had been somehow lured into this wilderness. He'd never heard anything from the rest of them. It was clear that they had never, ever come out. It must be like trying to swim in air.
He gathered intellectual energy for one last leap. "Maybe we're looking at this problem from the wrong end of the telescope."
The holy man revived a bit. "In what sense?"
"Maybe it's not about us at all. Maybe it was never about us. Maybe we would get somewhere useful if we tried to think hard about her. Let her be the center of this issue. Not us. Her. She's a two-hundred-million-year-old entity screaming at fo
ur hundred million stars. That's rather remarkable, on the face of it, isn't it?"
"I suppose."
"Then maybe it's her story. From her perspective, it all appears differently. She's not our 'victim'—she doesn't know about us at all. Within her own state of being, she is her own heroine. She is singing to those stars. Being human, we conceive of her as some rattletrap contraption we built, a prisoner in our dungeon—but maybe she's a pretty, young girl in an ivory tower. Because see, she's singing."
"That's like a tale for small children."
"So is your tale, your holiness. They are two different tales. But since we're not of her order of being, we're projecting our anthropomorphic interpretations. And we lack any sound method to distinguish your dark, evil, thoroughly depressing story from my romantic, light-hearted, wistful hypothesis."
"We do agree that the system manifests seriously aberrant behavior. She has destructive fits."
"She's just young."
"You've lost the thread. It's the aberrancy that has real-world implications. We'll never be able to judge the interior state of that system."
"Yes it is, I agree with that, too, but—what if someone else hears her cries? Not us humans. I mean entities like herself. What if she's speaking to them right now? Exchanging light with them! They might even be coming here. No human can ever move from star to star. Our lives are just too brief, the distances too great. But someone like her . . . if it took them thirty thousand years to travel over here, that's like a summer afternoon."
"An interstellar monster coming here to take a terrible vengeance on us?"
"No, no, you can't know that! It's all metaphorical! You think we're evil because you think humanity matters in this universe! And yes, to us, she seems ancient and awesome—but maybe, by the standards of her own kind, she is just a kid. A young, naïve girl calling out for some company. Sure—maybe some wicked stranger would come all the way out here just to kill her, exterminate us, and burn her home. Or maybe—maybe someone might venture here for love and understanding."
The holy man scratched at a fang. "For love. For sentiment? Emotion? No one talks much about 'artificial emotion.'"
Eclipse One Page 27