by Greg Egan
“To me, this was self-evidently false. What point could there be, I argued, in even conversing with a materialist? By his own admission, the words that came out of his mouth would be the result of nothing but a mindless, mechanical process! By his own theory, he could have no reason to think that those words would be the truth! Only believers in a transcendent human soul could claim any interest in the truth.”
Hamilton nodded slowly, a penitent’s gesture. “I was wrong, and I was put in my place. This might be self-evident to me, and it might be self-evident to you, but it’s certainly not what philosophers call an ‘analytical truth’: it’s not actually a nonsense, a contradiction in terms, to believe that we are mere machines. There might, there just might, be some reason why the words that emerge from a materialist’s mouth are truthful, despite their origins lying entirely in unthinking matter.
“There might.” Hamilton smiled wistfully. “I had to concede that possibility, because I only had my instinct, my gut feeling, to tell me otherwise.
“But the reason I only had my instinct to guide me was because I’d failed to learn of an event that had taken place many years before. A discovery made in 1930, by an Austrian mathematician named Kurt Gödel.”
Robert felt a shiver of excitement run down his spine. He’d been afraid that the whole contest would degenerate into theology, with Hamilton invoking Aquinas all night – or Aristotle, at best. But it looked as if his mysterious adviser had dragged him into the twentieth century, and they were going to have a chance to debate the real issues after all.
“What is it that we know Professor Stoney’s computers can do, and do well?” Hamilton continued. “Arithmetic! In a fraction of a second, they can add up a million numbers. Once we’ve told them, very precisely, what calculations to perform, they’ll complete them in the blink of an eye – even if those calculations would take you or me a lifetime.
“But do these machines understand what it is they’re doing? Professor Stoney says, ‘Not yet. Not right now. Give them time. Rome wasn’t built in a day.’” Hamilton nodded thoughtfully. “Perhaps that’s fair. His computers are only a few years old. They’re just babies. Why should they understand anything, so soon?
“But let’s stop and think about this a bit more carefully. A computer, as it stands today, is simply a machine that does arithmetic, and Professor Stoney isn’t proposing that they’re going to sprout new kinds of brains all on their own. Nor is he proposing giving them anything really new. He can already let them look at the world with television cameras, turning the pictures into a stream of numbers describing the brightness of different points on the screen … on which the computer can then perform arithmetic. He can already let them speak to us with a special kind of loudspeaker, to which the computer feeds a stream of numbers to describe how loud the sound should be … a stream of numbers produced by more arithmetic.
“So the world can come into the computer, as numbers, and words can emerge, as numbers too. All Professor Stoney hopes to add to his computers is a ‘cleverer’ way to do the arithmetic that takes the first set of numbers and churns out the second. It’s that ‘clever arithmetic’, he tells us, that will make these machines think.”
Hamilton folded his arms and paused for a moment. “What are we to make of this? Can doing arithmetic, and nothing more, be enough to let a machine understand anything? My instinct certainly tells me no, but who am I that you should trust my instinct?
“So, let’s narrow down the question of understanding, and to be scrupulously fair, let’s put it in the most favorable light possible for Professor Stoney. If there’s one thing a computer ought to be able to understand – as well as us, if not better – it’s arithmetic itself. If a computer could think at all, it would surely be able to grasp the nature of its own best talent.
“The question, then, comes down to this: can you describe all of arithmetic, using nothing but arithmetic? Thirty years ago – long before Professor Stoney and his computers came along – Professor Gödel asked himself exactly that question.
“Now, you might be wondering how anyone could even begin to describe the rules of arithmetic, using nothing but arithmetic itself.” Hamilton turned to the blackboard, picked up the chalk, and wrote two lines:
If x+z = y+z
then x = y
“This is an important rule, but it’s written in symbols, not numbers, because it has to be true for every number, every x, y and z. But Professor Gödel had a clever idea: why not use a code, like spies use, where every symbol is assigned a number?” Hamilton wrote:
The code for “a” is 1.
The code for “b” is 2.
“And so on. You can have a code for every letter of the alphabet, and for all the other symbols needed for arithmetic: plus signs, equals signs, that kind of thing. Telegrams are sent this way every day, with a code called the Baudot code, so there’s really nothing strange or sinister about it.
“All the rules of arithmetic that we learned at school can be written with a carefully chosen set of symbols, which can then be translated into numbers. Every question as to what does or does not follow from those rules can then be seen anew, as a question about numbers. If this line follows from this one,” Hamilton indicated the two lines of the cancellation rule, “we can see it in the relationship between their code numbers. We can judge each inference, and declare it valid or not, purely by doing arithmetic.
“So, given any proposition at all about arithmetic – such as the claim that ‘there are infinitely many prime numbers’ – we can restate the notion that we have a proof for that claim in terms of code numbers. If the code number for our claim is x, we can say ‘There is a number p, ending with the code number x, that passes our test for being the code number of a valid proof.’”
Hamilton took a visible breath.
“In 1930, Professor Gödel used this scheme to do something rather ingenious.” He wrote on the blackboard:
There DOES NOT EXIST a number p meeting the following condition: p is the code number of a valid proof of this claim.
“Here is a claim about arithmetic, about numbers. It has to be either true or false. So let’s start by supposing that it happens to be true. Then there is no number p that is the code number for a proof of this claim. So this is a true statement about arithmetic, but it can’t be proved merely by doing arithmetic!”
Hamilton smiled. “If you don’t catch on immediately, don’t worry; when I first heard this argument from a young friend of mine, it took a while for the meaning to sink in. But remember: the only hope a computer has for understanding anything is by doing arithmetic, and we’ve just found a statement that cannot be proved with mere arithmetic.
“Is this statement really true, though? We mustn’t jump to conclusions, we mustn’t damn the machines too hastily. Suppose this claim is false! Since it claims there is no number p that is the code number of its own proof, to be false there would have to be such a number, after all. And that number would encode the ‘proof’ of an acknowledged falsehood!”
Hamilton spread his arms triumphantly. “You and I, like every schoolboy, know that you can’t prove a falsehood from sound premises – and if the premises of arithmetic aren’t sound, what is? So we know, as a matter of certainty, that this statement is true.
“Professor Gödel was the first to see this, but with a little help and perseverance, any educated person can follow in his footsteps. A machine could never do that. We might divulge to a machine our own knowledge of this fact, offering it as something to be taken on trust, but the machine could neither stumble on this truth for itself, nor truly comprehend it when we offered it as a gift.
“You and I understand arithmetic, in a way that no electronic calculator ever will. What hope has a machine, then, of moving beyond its own most favorable milieu and comprehending any wider truth?
“None at all, ladies and gentlemen. Though this detour into mathematics might have seemed arcane to you, it has served a very down-to-Earth purpose.
It has proved – beyond refutation by even the most ardent materialist or the most pedantic philosopher – what we common folk knew all along: no machine will ever think.”
Hamilton took his seat. For a moment, Robert was simply exhilarated; coached or not, Hamilton had grasped the essential features of the incompleteness proof, and presented them to a lay audience. What might have been a night of shadow-boxing – with no blows connecting, and nothing for the audience to judge but two solo performances in separate arenas – had turned into a genuine clash of ideas.
As Polanyi introduced him and he walked to the podium, Robert realized that his usual shyness and self-consciousness had evaporated. He was filled with an altogether different kind of tension: he sensed more acutely than ever what was at stake.
When he reached the podium, he adopted the posture of someone about to begin a prepared speech, but then he caught himself, as if he’d forgotten something. “Bear with me for a moment.” He walked around to the far side of the blackboard and quickly wrote a few words on it, upside-down. Then he resumed his place.
“Can a machine think? Professor Hamilton would like us to believe that he’s settled the issue once and for all, by coming up with a statement that we know is true, but a particular machine – programmed to explore the theorems of arithmetic in a certain rigid way – would never be able to produce. Well … we all have our limitations.” He flipped the blackboard over to reveal what he’d written on the opposite side:
If Robert Stoney speaks these words, he will NOT be telling the truth.
He waited a few beats, then continued.
“What I’d like to explore, though, is not so much a question of limitations, as of opportunities. How exactly is it that we’ve all ended up with this mysterious ability to know that Gödel’s statement is true? Where does this advantage, this great insight, come from? From our souls? From some immaterial entity that no machine could ever possess? Is that the only possible source, the only conceivable explanation? Or might it come from something a little less ethereal?
“As Professor Hamilton explained, we believe Gödel’s statement is true because we trust the rules of arithmetic not to lead us into contradictions and falsehoods. But where does that trust come from? How does it arise?”
Robert turned the blackboard back to Hamilton’s side, and pointed to the cancellation rule. “If x plus z equals y plus z, then x equals y. Why is this so reasonable? We might not learn to put it quite like this until we’re in our teens, but if you showed a young child two boxes – without revealing their contents – added an equal number of shells, or stones, or pieces of fruit to both, and then let the child look inside to see that each box now contained the same number of items, it wouldn’t take any formal education for the child to understand that the two boxes must have held the same number of things to begin with.
“The child knows, we all know, how a certain kind of object behaves. Our lives are steeped in direct experience of whole numbers: whole numbers of coins, stamps, pebbles, birds, cats, sheep, buses. If I tried to persuade a six-year-old that I could put three stones in a box, remove one of them, and be left with four … he’d simply laugh at me. Why? It’s not merely that he’s sure to have taken one thing away from three to get two, on many prior occasions. Even a child understands that some things that appear reliable will eventually fail: a toy that works perfectly, day after day, for a month or a year, can still break. But not arithmetic, not taking one from three. He can’t even picture that failing. Once you’ve lived in the world, once you’ve seen how it works, the failure of arithmetic becomes unimaginable.
“Professor Hamilton suggests that this is down to our souls. But what would he say about a child reared in a world of water and mist, never in the company of more than one person at a time, never taught to count on his fingers and toes. I doubt that such a child would possess the same certainty that you and I have, as to the impossibility of arithmetic ever leading him astray. To banish whole numbers entirely from his world would require very strange surroundings, and a level of deprivation amounting to cruelty, but would that be enough to rob a child of his soul?
“A computer, programmed to pursue arithmetic as Professor Hamilton has described, is subject to far more deprivation than that child. If I’d been raised with my hands and feet tied, my head in a sack, and someone shouting orders at me, I doubt that I’d have much grasp of reality – and I’d still be better prepared for the task than such a computer. It’s a great mercy that a machine treated that way wouldn’t be able to think: if it could, the shackles we’d placed upon it would be criminally oppressive.
“But that’s hardly the fault of the computer, or a revelation of some irreparable flaw in its nature. If we want to judge the potential of our machines with any degree of honesty, we have to play fair with them, not saddle them with restrictions that we’d never dream of imposing on ourselves. There really is no point comparing an eagle with a spanner, or a gazelle with a washing machine: it’s our jets that fly and our cars that run, albeit in quite different ways than any animal.
“Thought is sure to be far harder to achieve than those other skills, and to do so we might need to mimic the natural world far more closely. But I believe that once a machine is endowed with facilities resembling the inborn tools for learning that we all have as our birthright, and is set free to learn the way a child learns, through experience, observation, trial and error, hunches and failures – instead of being handed a list of instructions that it has no choice but to obey – we will finally be in a position to compare like with like.
“When that happens, and we can meet and talk and argue with these machines – about arithmetic, or any other topic – there’ll be no need to take the word of Professor Gödel, or Professor Hamilton, or myself, for anything. We’ll invite them down to the local pub, and interrogate them in person. And if we play fair with them, we’ll use the same experience and judgment we use with any friend, or guest, or stranger, to decide for ourselves whether or not they can think.”
#
The BBC put on a lavish assortment of wine and cheese in a small room off the studio. Robert ended up in a heated argument with Polanyi, who revealed himself to be firmly on the negative side, while Helen flirted shamelessly with Hamilton’s young friend, who turned out to have a PhD in algebraic geometry from Cambridge; he must have completed the degree just before Robert had come back from Manchester. After exchanging some polite formalities with Hamilton, Robert kept his distance, sensing that any further contact would not be welcome.
An hour later, though, after getting lost in the maze of corridors on his way back from the toilets, Robert came across Hamilton sitting alone in the studio, weeping.
He almost backed away in silence, but Hamilton looked up and saw him. With their eyes locked, it was impossible to retreat.
Robert said, “It’s your wife?” He’d heard that she’d been seriously ill, but the gossip had included a miraculous recovery. Some friend of the family had laid hands on her a year ago, and she’d gone into remission.
Hamilton said, “She’s dying.”
Robert approached and sat beside him. “From what?”
“Breast cancer. It’s spread throughout her body. Into her bones, into her lungs, into her liver.” He sobbed again, a helpless spasm, then caught himself angrily. “Suffering is the chisel God uses to shape us. What kind of idiot comes up with a line like that?”
Robert said, “I’ll talk to a friend of mine, an oncologist at Guy’s Hospital. He’s doing a trial of a new genetic treatment.”
Hamilton stared at him. “One of your miracle cures?”
“No, no. I mean, only very indirectly.”
Hamilton said angrily, “She won’t take your poison.”
Robert almost snapped back: She won’t? Or you won’t let her? But it was an unfair question. In some marriages, the lines blurred. It was not for him to judge the way the two of them faced this together.
“They go away in order to be wit
h us in a new way, even closer than before.” Hamilton spoke the words like a defiant incantation, a declaration of faith that would ward off temptation, whether or not he entirely believed it.
Robert was silent for a while, then he said, “I lost someone close to me, when I was a boy. And I thought the same thing. I thought he was still with me, for a long time afterward. Guiding me. Encouraging me.” It was hard to get the words out; he hadn’t spoken about this to anyone for almost thirty years. “I dreamed up a whole theory to explain it, in which ‘souls’ used quantum uncertainty to control the body during life, and communicate with the living after death, without breaking any laws of physics. The kind of thing every science-minded seventeen-year-old probably stumbles on, and takes seriously for a couple of weeks, before realizing how nonsensical it is. But I had a good reason not to see the flaws, so I clung to it for almost two years. Because I missed him so much, it took me that long to understand what I was doing, how I was deceiving myself.”
Hamilton said pointedly, “If you’d not tried to explain it, you might never have lost him. He might still be with you now.”
Robert thought about this. “I’m glad he’s not, though. It wouldn’t be fair on either of us.”
Hamilton shuddered. “Then you can’t have loved him very much, can you?” He put his head in his arms. “Just fuck off, now, will you.”
Robert said, “What exactly would it take, to prove to you that I’m not in league with the devil?”
Hamilton turned red eyes on him and announced triumphantly, “Nothing will do that! I saw what happened to Quint’s gun!”
Robert sighed. “That was a conjuring trick. Stage magic, not black magic.”
“Oh yes? Show me how it’s done, then. Teach me how to do it, so I can impress my friends.”
“It’s rather technical. It would take all night.”