Rationality- From AI to Zombies

Home > Science > Rationality- From AI to Zombies > Page 61
Rationality- From AI to Zombies Page 61

by Eliezer Yudkowsky


  But that would be mere guessing. It wouldn’t be, y’know, absolutely and eternally certain. The Greek philosophers—like most prescientific philosophers—were rather fond of certainty.

  Luckily the Greek philosophers have a crushing rejoinder to your questioning. You have misunderstood the meaning of “All humans are mortal,” they say. It is not a mere observation. It is part of the definition of the word “human.” Mortality is one of several properties that are individually necessary, and together sufficient, to determine membership in the class “human.” The statement “All humans are mortal” is a logically valid truth, absolutely unquestionable. And if Socrates is human, he must be mortal: it is a logical deduction, as certain as certain can be.

  But then we can never know for certain that Socrates is a “human” until after Socrates has been observed to be mortal. It does no good to observe that Socrates speaks fluent Greek, or that Socrates has red blood, or even that Socrates has human DNA. None of these characteristics are logically equivalent to mortality. You have to see him die before you can conclude that he was human.

  (And even then it’s not infinitely certain. What if Socrates rises from the grave a night after you see him die? Or more realistically, what if Socrates is signed up for cryonics? If mortality is defined to mean finite lifespan, then you can never really know if someone was human, until you’ve observed to the end of eternity—just to make sure they don’t come back. Or you could think you saw Socrates keel over, but it could be an illusion projected onto your eyes with a retinal scanner. Or maybe you just hallucinated the whole thing . . .)

  The problem with syllogisms is that they’re always valid. “All humans are mortal; Socrates is human; therefore Socrates is mortal” is—if you treat it as a logical syllogism—logically valid within our own universe. It’s also logically valid within neighboring Everett branches in which, due to a slightly different evolved biochemistry, hemlock is a delicious treat rather than a poison. And it’s logically valid even in universes where Socrates never existed, or for that matter, where humans never existed.

  The Bayesian definition of evidence favoring a hypothesis is evidence which we are more likely to see if the hypothesis is true than if it is false. Observing that a syllogism is logically valid can never be evidence favoring any empirical proposition, because the syllogism will be logically valid whether that proposition is true or false.

  Syllogisms are valid in all possible worlds, and therefore, observing their validity never tells us anything about which possible world we actually live in.

  This doesn’t mean that logic is useless—just that logic can only tell us that which, in some sense, we already know. But we do not always believe what we know. Is the number 29,384,209 prime? By virtue of how I define my decimal system and my axioms of arithmetic, I have already determined my answer to this question—but I do not know what my answer is yet, and I must do some logic to find out.

  Similarly, if I form the uncertain empirical generalization “Humans are vulnerable to hemlock,” and the uncertain empirical guess “Socrates is human,” logic can tell me that my previous guesses are predicting that Socrates will be vulnerable to hemlock.

  It’s been suggested that we can view logical reasoning as resolving our uncertainty about impossible possible worlds—eliminating probability mass in logically impossible worlds which we did not know to be logically impossible. In this sense, logical argument can be treated as observation.

  But when you talk about an empirical prediction like “Socrates is going to keel over and stop breathing” or “Socrates is going to do fifty jumping jacks and then compete in the Olympics next year,” that is a matter of possible worlds, not impossible possible worlds.

  Logic can tell us which hypotheses match up to which observations, and it can tell us what these hypotheses predict for the future—it can bring old observations and previous guesses to bear on a new problem. But logic never flatly says, “Socrates will stop breathing now.” Logic never dictates any empirical question; it never settles any real-world query which could, by any stretch of the imagination, go either way.

  Just remember the Litany Against Logic:

  Logic stays true, wherever you may go,

  So logic never tells you where you live.

  *

  155

  Words as Hidden Inferences

  Suppose I find a barrel, sealed at the top, but with a hole large enough for a hand. I reach in and feel a small, curved object. I pull the object out, and it’s blue—a bluish egg. Next I reach in and feel something hard and flat, with edges—which, when I extract it, proves to be a red cube. I pull out 11 eggs and 8 cubes, and every egg is blue, and every cube is red.

  Now I reach in and I feel another egg-shaped object. Before I pull it out and look, I have to guess: What will it look like?

  The evidence doesn’t prove that every egg in the barrel is blue and every cube is red. The evidence doesn’t even argue this all that strongly: 19 is not a large sample size. Nonetheless, I’ll guess that this egg-shaped object is blue—or as a runner-up guess, red. If I guess anything else, there’s as many possibilities as distinguishable colors—and for that matter, who says the egg has to be a single shade? Maybe it has a picture of a horse painted on.

  So I say “blue,” with a dutiful patina of humility. For I am a sophisticated rationalist-type person, and I keep track of my assumptions and dependencies—I guess, but I’m aware that I’m guessing . . . right?

  But when a large yellow striped feline-shaped object leaps out at me from the shadows, I think, “Yikes! A tiger!” Not, “Hm . . . objects with the properties of largeness, yellowness, stripedness, and feline shape, have previously often possessed the properties ‘hungry’ and ‘dangerous,’ and thus, although it is not logically necessary, it may be an empirically good guess that aaauuughhhh CRUNCH CRUNCH GULP.”

  The human brain, for some odd reason, seems to have been adapted to make this inference quickly, automatically, and without keeping explicit track of its assumptions.

  And if I name the egg-shaped objects “bleggs” (for blue eggs) and the red cubes “rubes,” then, when I reach in and feel another egg-shaped object, I may think, Oh, it’s a blegg, rather than considering all that problem-of-induction stuff.

  It is a common misconception that you can define a word any way you like.

  This would be true if the brain treated words as purely logical constructs, Aristotelian classes, and you never took out any more information than you put in.

  Yet the brain goes on about its work of categorization, whether or not we consciously approve. “All humans are mortal; Socrates is a human; therefore Socrates is mortal”—thus spake the ancient Greek philosophers. Well, if mortality is part of your logical definition of “human,” you can’t logically classify Socrates as human until you observe him to be mortal. But—this is the problem—Aristotle knew perfectly well that Socrates was a human. Aristotle’s brain placed Socrates in the “human” category as efficiently as your own brain categorizes tigers, apples, and everything else in its environment: Swiftly, silently, and without conscious approval.

  Aristotle laid down rules under which no one could conclude Socrates was “human” until after he died. Nonetheless, Aristotle and his students went on concluding that living people were humans and therefore mortal; they saw distinguishing properties such as human faces and human bodies, and their brains made the leap to inferred properties such as mortality.

  Misunderstanding the working of your own mind does not, thankfully, prevent the mind from doing its work. Otherwise Aristotelians would have starved, unable to conclude that an object was edible merely because it looked and felt like a banana.

  So the Aristotelians went on classifying environmental objects on the basis of partial information, the way people had always done. Students of Aristotelian logic went on thinking exactly the same way, but they had acquired an erroneous picture of what they were doing.

  If you asked an Aristote
lian philosopher whether Carol the grocer was mortal, they would say “Yes.” If you asked them how they knew, they would say “All humans are mortal; Carol is human; therefore Carol is mortal.” Ask them whether it was a guess or a certainty, and they would say it was a certainty (if you asked before the sixteenth century, at least). Ask them how they knew that humans were mortal, and they would say it was established by definition.

  The Aristotelians were still the same people, they retained their original natures, but they had acquired incorrect beliefs about their own functioning. They looked into the mirror of self-awareness, and saw something unlike their true selves: they reflected incorrectly.

  Your brain doesn’t treat words as logical definitions with no empirical consequences, and so neither should you. The mere act of creating a word can cause your mind to allocate a category, and thereby trigger unconscious inferences of similarity. Or block inferences of similarity; if I create two labels I can get your mind to allocate two categories. Notice how I said “you” and “your brain” as if they were different things?

  Making errors about the inside of your head doesn’t change what’s there; otherwise Aristotle would have died when he concluded that the brain was an organ for cooling the blood. Philosophical mistakes usually don’t interfere with blink-of-an-eye perceptual inferences.

  But philosophical mistakes can severely mess up the deliberate thinking processes that we use to try to correct our first impressions. If you believe that you can “define a word any way you like,” without realizing that your brain goes on categorizing without your conscious oversight, then you won’t make the effort to choose your definitions wisely.

  *

  156

  Extensions and Intensions

  “What is red?”

  “Red is a color.”

  “What’s a color?”

  “A color is a property of a thing.”

  But what is a thing? And what’s a property? Soon the two are lost in a maze of words defined in other words, the problem that Steven Harnad once described as trying to learn Chinese from a Chinese/Chinese dictionary.

  Alternatively, if you asked me “What is red?” I could point to a stop sign, then to someone wearing a red shirt, and a traffic light that happens to be red, and blood from where I accidentally cut myself, and a red business card, and then I could call up a color wheel on my computer and move the cursor to the red area. This would probably be sufficient, though if you know what the word “No” means, the truly strict would insist that I point to the sky and say “No.”

  I think I stole this example from S. I. Hayakawa—though I’m really not sure, because I heard this way back in the indistinct blur of my childhood. (When I was twelve, my father accidentally deleted all my computer files. I have no memory of anything before that.)

  But that’s how I remember first learning about the difference between intensional and extensional definition. To give an “intensional definition” is to define a word or phrase in terms of other words, as a dictionary does. To give an “extensional definition” is to point to examples, as adults do when teaching children. The preceding sentence gives an intensional definition of “extensional definition,” which makes it an extensional example of “intensional definition.”

  In Hollywood Rationality and popular culture generally, “rationalists” are depicted as word-obsessed, floating in endless verbal space disconnected from reality.

  But the actual Traditional Rationalists have long insisted on maintaining a tight connection to experience:

  If you look into a textbook of chemistry for a definition of lithium, you may be told that it is that element whose atomic weight is 7 very nearly. But if the author has a more logical mind he will tell you that if you search among minerals that are vitreous, translucent, grey or white, very hard, brittle, and insoluble, for one which imparts a crimson tinge to an unluminous flame, this mineral being triturated with lime or witherite rats-bane, and then fused, can be partly dissolved in muriatic acid; and if this solution be evaporated, and the residue be extracted with sulphuric acid, and duly purified, it can be converted by ordinary methods into a chloride, which being obtained in the solid state, fused, and electrolyzed with half a dozen powerful cells, will yield a globule of a pinkish silvery metal that will float on gasolene; and the material of that is a specimen of lithium.

  —Charles Sanders Peirce1

  That’s an example of “logical mind” as described by a genuine Traditional Rationalist, rather than a Hollywood scriptwriter.

  But note: Peirce isn’t actually showing you a piece of lithium. He didn’t have pieces of lithium stapled to his book. Rather he’s giving you a treasure map—an intensionally defined procedure which, when executed, will lead you to an extensional example of lithium. This is not the same as just tossing you a hunk of lithium, but it’s not the same as saying “atomic weight 7” either. (Though if you had sufficiently sharp eyes, saying “3 protons” might let you pick out lithium at a glance . . .)

  So that is intensional and extensional definition, which is a way of telling someone else what you mean by a concept. When I talked about “definitions” above, I talked about a way of communicating concepts—telling someone else what you mean by “red,” “tiger,” “human,” or “lithium.” Now let’s talk about the actual concepts themselves.

  The actual intension of my “tiger” concept would be the neural pattern (in my temporal cortex) that inspects an incoming signal from the visual cortex to determine whether or not it is a tiger.

  The actual extension of my “tiger” concept is everything I call a tiger.

  Intensional definitions don’t capture entire intensions; extensional definitions don’t capture entire extensions. If I point to just one tiger and say the word “tiger,” the communication may fail if they think I mean “dangerous animal” or “male tiger” or “yellow thing.” Similarly, if I say “dangerous yellow-black striped animal,” without pointing to anything, the listener may visualize giant hornets.

  You can’t capture in words all the details of the cognitive concept—as it exists in your mind—that lets you recognize things as tigers or nontigers. It’s too large. And you can’t point to all the tigers you’ve ever seen, let alone everything you would call a tiger.

  The strongest definitions use a crossfire of intensional and extensional communication to nail down a concept. Even so, you only communicate maps to concepts, or instructions for building concepts—you don’t communicate the actual categories as they exist in your mind or in the world.

  (Yes, with enough creativity you can construct exceptions to this rule, like “Sentences Eliezer Yudkowsky has published containing the term ‘huragaloni’ as of Feb 4, 2008.” I’ve just shown you this concept’s entire extension. But except in mathematics, definitions are usually treasure maps, not treasure.)

  So that’s another reason you can’t “define a word any way you like”: You can’t directly program concepts into someone else’s brain.

  Even within the Aristotelian paradigm, where we pretend that the definitions are the actual concepts, you don’t have simultaneous freedom of intension and extension. Suppose I define Mars as “A huge red rocky sphere, around a tenth of Earth’s mass and 50% further away from the Sun.” It’s then a separate matter to show that this intensional definition matches some particular extensional thing in my experience, or indeed, that it matches any real thing whatsoever. If instead I say “That’s Mars” and point to a red light in the night sky, it becomes a separate matter to show that this extensional light matches any particular intensional definition I may propose—or any intensional beliefs I may have—such as “Mars is the God of War.”

  But most of the brain’s work of applying intensions happens sub-deliberately. We aren’t consciously aware that our identification of a red light as “Mars” is a separate matter from our verbal definition “Mars is the God of War.” No matter what kind of intensional definition I make up to describe Mars, my mind believes th
at “Mars” refers to this thingy, and that it is the fourth planet in the Solar System.

  When you take into account the way the human mind actually, pragmatically works, the notion “I can define a word any way I like” soon becomes “I can believe anything I want about a fixed set of objects” or “I can move any object I want in or out of a fixed membership test.” Just as you can’t usually convey a concept’s whole intension in words because it’s a big complicated neural membership test, you can’t control the concept’s entire intension because it’s applied sub-deliberately. This is why arguing that XYZ is true “by definition” is so popular. If definition changes behaved like the empirical null-ops they’re supposed to be, no one would bother arguing them. But abuse definitions just a little, and they turn into magic wands—in arguments, of course; not in reality.

  *

  1. Charles Sanders Peirce, Collected Papers (Harvard University Press, 1931).

  157

  Similarity Clusters

  Once upon a time, the philosophers of Plato’s Academy claimed that the best definition of human was a “featherless biped.” Diogenes of Sinope, also called Diogenes the Cynic, is said to have promptly exhibited a plucked chicken and declared “Here is Plato’s man.” The Platonists promptly changed their definition to “a featherless biped with broad nails.”

  No dictionary, no encyclopedia, has ever listed all the things that humans have in common. We have red blood, five fingers on each of two hands, bony skulls, 23 pairs of chromosomes—but the same might be said of other animal species. We make complex tools to make complex tools, we use syntactical combinatorial language, we harness critical fission reactions as a source of energy: these things may serve out to single out only humans, but not all humans—many of us have never built a fission reactor. With the right set of necessary-and-sufficient gene sequences you could single out all humans, and only humans—at least for now—but it would still be far from all that humans have in common.

 

‹ Prev