Rationality- From AI to Zombies

by Eliezer Yudkowsky

  This certainly seems like the inner listener is being caught in the act of listening by whatever part of you writes the internal narrative and flaps your tongue.

  Imagine that a mysterious race of aliens visit you, and leave you a mysterious black box as a gift. You try poking and prodding the black box, but (as far as you can tell) you never succeed in eliciting a reaction. You can’t make the black box produce gold coins or answer questions. So you conclude that the black box is causally inactive: “For all X, the black box doesn’t do X.” The black box is an effect, but not a cause; epiphenomenal; without causal potency. In your mind, you test this general hypothesis to see if it is true in some trial cases, and it seems to be true—“Does the black box turn lead to gold? No. Does the black box boil water? No.”

  But you can see the black box; it absorbs light, and weighs heavy in your hand. This, too, is part of the dance of causality. If the black box were wholly outside the causal universe, you couldn’t see it; you would have no way to know it existed; you could not say, “Thanks for the black box.” You didn’t think of this counterexample, when you formulated the general rule: “All X: Black box doesn’t do X.” But it was there all along.

  (Actually, the aliens left you another black box, this one purely epiphenomenal, and you haven’t the slightest clue that it’s there in your living room. That was their joke.)

  If you can close your eyes, and sense yourself sensing—if you can be aware of yourself being aware, and think “I am aware that I am aware”—and say out loud, “I am aware that I am aware”—then your consciousness is not without effect on your internal narrative, or your moving lips. You can see yourself seeing, and your internal narrative reflects this, and so do your lips if you choose to say it out loud.

  I have not seen the above argument written out that particular way—“the listener caught in the act of listening”—though it may well have been said before.

  But it is a standard point—which zombie-ist philosophers accept!—that the Zombie World’s philosophers, being atom-by-atom identical to our own philosophers, write identical papers about the philosophy of consciousness.

  At this point, the Zombie World stops being an intuitive consequence of the idea of a passive listener.

  Philosophers writing papers about consciousness would seem to be at least one effect of consciousness upon the world. You can argue clever reasons why this is not so, but you have to be clever.

  You would intuitively suppose that if your inward awareness went away, this would change the world, in that your internal narrative would no longer say things like “There is a mysterious listener within me,” because the mysterious listener would be gone. It is usually right after you focus your awareness on your awareness, that your internal narrative says “I am aware of my awareness,” which suggests that if the first event never happened again, neither would the second. You can argue clever reasons why this is not so, but you have to be clever.

  You can form a propositional belief that “Consciousness is without effect,” and not see any contradiction at first, if you don’t realize that talking about consciousness is an effect of being conscious. But once you see the connection from the general rule that consciousness has no effect, to the specific implication that consciousness has no effect on how philosophers write papers about consciousness, zombie-ism stops being intuitive and starts requiring you to postulate strange things.

  One strange thing you might postulate is that there’s a Zombie Master, a god within the Zombie World who surreptitiously takes control of zombie philosophers and makes them talk and write about consciousness.

  A Zombie Master doesn’t seem impossible. Human beings often don’t sound all that coherent when talking about consciousness. It might not be that hard to fake their discourse, to the standards of, say, a human amateur talking in a bar. Maybe you could take, as a corpus, one thousand human amateurs trying to discuss consciousness; feed them into a non-conscious but sophisticated AI, better than today’s models but not self-modifying; and get back discourse about “consciousness” that sounded as sensible as most humans, which is to say, not very.

  But this speech about “consciousness” would not be spontaneous. It would not be produced within the AI. It would be a recorded imitation of someone else talking. That is just a holodeck, with a central AI writing the speech of the non-player characters. This is not what the Zombie World is about.

  By supposition, the Zombie World is atom-by-atom identical to our own, except that the inhabitants lack consciousness. Furthermore, the atoms in the Zombie World move under the same laws of physics as in our own world. If there are “bridging laws” that govern which configurations of atoms evoke consciousness, those bridging laws are absent. But, by hypothesis, the difference is not experimentally detectable. When it comes to saying whether a quark zigs or zags or exerts a force on nearby quarks—anything experimentally measurable—the same physical laws govern.

  The Zombie World has no room for a Zombie Master, because a Zombie Master has to control the zombie’s lips, and that control is, in principle, experimentally detectable. The Zombie Master moves lips, therefore it has observable consequences. There would be a point where an electron zags, instead of zigging, because the Zombie Master says so. (Unless the Zombie Master is actually in the world, as a pattern of quarks—but then the Zombie World is not atom-by-atom identical to our own, unless you think this world also contains a Zombie Master.)

  When a philosopher in our world types, “I think the Zombie World is possible,” their fingers strike keys in sequence: Z-O-M-B-I-E. There is a chain of causality that can be traced back from these keystrokes: muscles contracting, nerves firing, commands sent down through the spinal cord, from the motor cortex—and then into less understood areas of the brain, where the philosopher’s internal narrative first began talking about “consciousness.”

  And the philosopher’s zombie twin strikes the same keys, for the same reason, causally speaking. There is no cause within the chain of explanation for why the philosopher writes the way they do that is not also present in the zombie twin. The zombie twin also has an internal narrative about “consciousness,” that a super-fMRI could read out of the auditory cortex. And whatever other thoughts, or other causes of any kind, led to that internal narrative, they are exactly the same in our own universe and in the Zombie World.

  So you can’t say that the philosopher is writing about consciousness because of consciousness, while the zombie twin is writing about consciousness because of a Zombie Master or AI chatbot. When you trace back the chain of causality behind the keyboard, to the internal narrative echoed in the auditory cortex, to the cause of the narrative, you must find the same physical explanation in our world as in the zombie world.

  As the most formidable advocate of zombie-ism, David Chalmers, writes:1

  Think of my zombie twin in the universe next door. He talks about conscious experience all the time—in fact, he seems obsessed by it. He spends ridiculous amounts of time hunched over a computer, writing chapter after chapter on the mysteries of consciousness. He often comments on the pleasure he gets from certain sensory qualia, professing a particular love for deep greens and purples. He frequently gets into arguments with zombie materialists, arguing that their position cannot do justice to the realities of conscious experience.

  And yet he has no conscious experience at all! In his universe, the materialists are right and he is wrong. Most of his claims about conscious experience are utterly false. But there is certainly a physical or functional explanation of why he makes the claims he makes. After all, his universe is fully law-governed, and no events therein are miraculous, so there must be some explanation of his claims.

  . . . Any explanation of my twin’s behavior will equally count as an explanation of my behavior, as the processes inside his body are precisely mirrored by those inside mine. The explanation of his claims obviously does not depend on the existence of consciousness, as there is no consciousness in his wor
ld. It follows that the explanation of my claims is also independent of the existence of consciousness.

  Chalmers is not arguing against zombies; those are his actual beliefs!

  This paradoxical situation is at once delightful and disturbing. It is not obviously fatal to the nonreductive position, but it is at least something that we need to come to grips with . . .

  I would seriously nominate this as the largest bullet ever bitten in the history of time. And that is a backhanded compliment to David Chalmers: A lesser mortal would simply fail to see the implications, or refuse to face them, or rationalize a reason it wasn’t so.

  Why would anyone bite a bullet that large? Why would anyone postulate unconscious zombies who write papers about consciousness for exactly the same reason that our own genuinely conscious philosophers do?

  Not because of the first intuition I wrote about, the intuition of the passive listener. That intuition may say that zombies can drive cars or do math or even fall in love, but it doesn’t say that zombies write philosophy papers about their passive listeners.

  The zombie argument does not rest solely on the intuition of the passive listener. If this was all there was to the zombie argument, it would be dead by now, I think. The intuition that the “listener” can be eliminated without effect would go away as soon as you realized that your internal narrative routinely seems to catch the listener in the act of listening.

  No, the drive to bite this bullet comes from an entirely different intuition—the intuition that no matter how many atoms you add up, no matter how many masses and electrical charges interact with each other, they will never necessarily produce a subjective sensation of the mysterious redness of red. It may be a fact about our physical universe (Chalmers says) that putting such-and-such atoms into such-and-such a position evokes a sensation of redness; but if so, it is not a necessary fact, it is something to be explained above and beyond the motion of the atoms.

  But if you consider the second intuition on its own, without the intuition of the passive listener, it is hard to see why it implies zombie-ism. Maybe there’s just a different kind of stuff, apart from and additional to atoms, that is not causally passive—a soul that actually does stuff, a soul that plays a real causal role in why we write about “the mysterious redness of red.” Take out the soul, and . . . well, assuming you don’t just fall over in a coma, you certainly won’t write any more papers about consciousness!

  This is the position taken by Descartes and most other ancient thinkers: The soul is of a different kind, but it interacts with the body. Descartes’s position is technically known as substance dualism—there is a thought-stuff, a mind-stuff, and it is not like atoms; but it is causally potent, interactive, and leaves a visible mark on our universe.

  Zombie-ists are property dualists—they don’t believe in a separate soul; they believe that matter in our universe has additional properties beyond the physical.

  “Beyond the physical”? What does that mean? It means the extra properties are there, but they don’t influence the motion of the atoms, like the properties of electrical charge or mass. The extra properties are not experimentally detectable by third parties; you know you are conscious, from the inside of your extra properties, but no scientist can ever directly detect this from outside.

  So the additional properties are there, but not causally active. The extra properties do not move atoms around, which is why they can’t be detected by third parties.

  And that’s why we can (allegedly) imagine a universe just like this one, with all the atoms in the same places, but the extra properties missing, so that everything goes on the same as before, but no one is conscious.

  The Zombie World may not be physically possible, say the zombie-ists—because it is a fact that all the matter in our universe has the extra properties, or obeys the bridging laws that evoke consciousness—but the Zombie World is logically possible: the bridging laws could have been different.

  But, once you realize that conceivability is not the same as logical possibility, and that the Zombie World isn’t even all that intuitive, why say that the Zombie World is logically possible?

  Why, oh why, say that the extra properties are epiphenomenal and indetectable?

  We can put this dilemma very sharply: Chalmers believes that there is something called consciousness, and this consciousness embodies the true and indescribable substance of the mysterious redness of red. It may be a property beyond mass and charge, but it’s there, and it is consciousness. Now, having said the above, Chalmers furthermore specifies that this true stuff of consciousness is epiphenomenal, without causal potency—but why say that?

  Why say that you could subtract this true stuff of consciousness, and leave all the atoms in the same place doing the same things? If that’s true, we need some separate physical explanation for why Chalmers talks about “the mysterious redness of red.” That is, there exists both a mysterious redness of red, which is extra-physical, and an entirely separate reason, within physics, why Chalmers talks about the “mysterious redness of red.”

  Chalmers does confess that these two things seem like they ought to be related, but really, why do you need both? Why not just pick one or the other?

  Once you’ve postulated that there is a mysterious redness of red, why not just say that it interacts with your internal narrative and makes you talk about the “mysterious redness of red”?

  Isn’t Descartes taking the simpler approach, here? The strictly simpler approach?

  Why postulate an extramaterial soul, and then postulate that the soul has no effect on the physical world, and then postulate a mysterious unknown material process that causes your internal narrative to talk about conscious experience?

  Why not postulate the true stuff of consciousness which no amount of mere mechanical atoms can add up to, and then, having gone that far already, let this true stuff of consciousness have causal effects like making philosophers talk about consciousness?

  I am not endorsing Descartes’s view. But at least I can understand where Descartes is coming from. Consciousness seems mysterious, so you postulate a mysterious stuff of consciousness. Fine.

  But now the zombie-ists postulate that this mysterious stuff doesn’t do anything, so you need a whole new explanation for why you say you’re conscious.

  That isn’t vitalism. That’s something so bizarre that vitalists would spit out their coffee. “When fires burn, they release phlogiston. But phlogiston doesn’t have any experimentally detectable impact on our universe, so you’ll have to go looking for a separate explanation of why a fire can melt snow.” What?

  Are property dualists under the impression that if they postulate a new active force, something that has a causal impact on observables, they will be sticking their necks out too far?

  Me, I’d say that if you postulate a mysterious, separate, additional, inherently mental property of consciousness, above and beyond positions and velocities, then, at that point, you have already stuck your neck out as far as it can go. To postulate this stuff of consciousness, and then further postulate that it doesn’t do anything—for the love of cute kittens, why?

  There isn’t even an obvious career motive. “Hi, I’m a philosopher of consciousness. My subject matter is the most important thing in the universe and I should get lots of funding? Well, it’s nice of you to say so, but actually the phenomenon I study doesn’t do anything whatsoever.” (Argument from career impact is not valid, but I say it to leave a line of retreat.)

  Chalmers critiques substance dualism on the grounds that it’s hard to see what new theory of physics, what new substance that interacts with matter, could possibly explain consciousness. But property dualism has exactly the same problem. No matter what kind of dual property you talk about, how exactly does it explain consciousness?

  When Chalmers postulated an extra property that is consciousness, he took that leap across the unexplainable. How does it help his theory to further specify that this extra property has no effect? W
hy not just let it be causal?

  If I were going to be unkind, this would be the time to drag in the dragon—to mention Carl Sagan’s parable of the dragon in the garage. “I have a dragon in my garage.” Great! I want to see it, let’s go! “You can’t see it—it’s an invisible dragon.” Oh, I’d like to hear it then. “Sorry, it’s an inaudible dragon.” I’d like to measure its carbon dioxide output. “It doesn’t breathe.” I’ll toss a bag of flour into the air, to outline its form. “The dragon is permeable to flour.”

  One motive for trying to make your theory unfalsifiable is that deep down you fear to put it to the test. Sir Roger Penrose (physicist) and Stuart Hameroff (neurologist) are substance dualists; they think that there is something mysterious going on in quantum, that Everett is wrong and that the “collapse of the wavefunction” is physically real, and that this is where consciousness lives and how it exerts causal effect upon your lips when you say aloud “I think therefore I am.” Believing this, they predicted that neurons would protect themselves from decoherence long enough to maintain macroscopic quantum states.

  This is in the process of being tested, and so far, prospects are not looking good for Penrose—

  —but Penrose’s basic conduct is scientifically respectable. Not Bayesian, maybe, but still fundamentally healthy. He came up with a wacky hypothesis. He said how to test it. He went out and tried to actually test it.

  As I once said to Stuart Hameroff, “I think the hypothesis you’re testing is completely hopeless, and your experiments should definitely be funded. Even if you don’t find exactly what you’re looking for, you’re looking in a place where no one else is looking, and you might find something interesting.”

  So a nasty dismissal of epiphenomenalism would be that zombie-ists are afraid to say the consciousness-stuff can have effects, because then scientists could go looking for the extra properties, and fail to find them.