Rationality- From AI to Zombies

Home > Science > Rationality- From AI to Zombies > Page 74
Rationality- From AI to Zombies Page 74

by Eliezer Yudkowsky


  * * *

  The right side of Bayes’s Theorem is derived from the left side through these steps:

  Once the derivation is finished, all the implications on the right side of the equation are of the form P(X|A) or P(X|¬A), while the implication on the left side is P(A|X). The symmetry arises because the elementary causal relations are generally implications from facts to observations, e.g., from breast cancer to positive mammography. The elementary steps in reasoning are generally implications from observations to facts, e.g., from a positive mammography to breast cancer. The left side of Bayes’s Theorem is an elementary inferential step from the observation of positive mammography to the conclusion of an increased probability of breast cancer. Implication is written right-to-left, so we write P(cancer|positive) on the left side of the equation. The right side of Bayes’s Theorem describes the elementary causal steps—for example, from breast cancer to a positive mammography—and so the implications on the right side of Bayes’s Theorem take the form P(positive|cancer) or P(positive|¬cancer).

  And that’s Bayes’s Theorem. Rational inference on the left end, physical causality on the right end; an equation with mind on one side and reality on the other. Remember how the scientific method turned out to be a special case of Bayes’s Theorem? If you wanted to put it poetically, you could say that Bayes’s Theorem binds reasoning into the physical universe.

  Okay, we’re done.

  Reverend Bayes says:

  You are now an initiate of the Bayesian Conspiracy.

  *

  1. Ward Casscells, Arno Schoenberger, and Thomas Graboys, “Interpretation by Physicians of Clinical Laboratory Results,” New England Journal of Medicine 299 (1978): 999–1001.

  2. David M. Eddy, “Probabilistic Reasoning in Clinical Medicine: Problems and Opportunities,” in Judgement Under Uncertainty: Heuristics and Biases, ed. Daniel Kahneman, Paul Slovic, and Amos Tversky (Cambridge University Press, 1982).

  3. Gerd Gigerenzer and Ulrich Hoffrage, “How to Improve Bayesian Reasoning without Instruction: Frequency Formats,” Psychological Review 102 (1995): 684–704.

  4. Ibid.

  5. Edwin T. Jaynes, “Probability Theory, with Applications in Science and Engineering,” Unpublished manuscript (1974).

  Book IV

  Mere Reality

  The World: An Introduction

  O. Lawful Truth

  181. Universal Fire

  182. Universal Law

  183. Is Reality Ugly?

  184. Beautiful Probability

  185. Outside the Laboratory

  186. The Second Law of Thermodynamics, and Engines of Cognition

  187. Perpetual Motion Beliefs

  188. Searching for Bayes-Structure

  P. Reductionism 101

  189. Dissolving the Question

  190. Wrong Questions

  191. Righting a Wrong Question

  192. Mind Projection Fallacy

  193. Probability is in the Mind

  194. The Quotation is Not the Referent

  195. Qualitatively Confused

  196. Think Like Reality

  197. Chaotic Inversion

  198. Reductionism

  199. Explaining vs. Explaining Away

  200. Fake Reductionism

  201. Savannah Poets

  Q. Joy in the Merely Real

  202. Joy in the Merely Real

  203. Joy in Discovery

  204. Bind Yourself to Reality

  205. If You Demand Magic, Magic Won’t Help

  206. Mundane Magic

  207. The Beauty of Settled Science

  208. Amazing Breakthrough Day: April 1st

  209. Is Humanism a Religion Substitute?

  210. Scarcity

  211. The Sacred Mundane

  212. To Spread Science, Keep It Secret

  213. Initiation Ceremony

  R. Physicalism 201

  214. Hand vs. Fingers

  215. Angry Atoms

  216. Heat vs. Motion

  217. Brain Breakthrough! It’s Made of Neurons!

  218. When Anthropomorphism Became Stupid

  219. A Priori

  220. Reductive Reference

  221. Zombies! Zombies?

  222. Zombie Responses

  223. The Generalized Anti-Zombie Principle

  224. GAZP vs. GLUT

  225. Belief in the Implied Invisible

  226. Zombies: The Movie

  227. Excluding the Supernatural

  228. Psychic Powers

  S. Quantum Physics and Many Worlds

  229. Quantum Explanations

  230. Configurations and Amplitude

  231. Joint Configurations

  232. Distinct Configurations

  233. Collapse Postulates

  234. Decoherence is Simple

  235. Decoherence is Falsifiable and Testable

  236. Privileging the Hypothesis

  237. Living in Many Worlds

  238. Quantum Non-Realism

  239. If Many-Worlds Had Come First

  240. Where Philosophy Meets Science

  241. Thou Art Physics

  242. Many Worlds, One Best Guess

  T. Science and Rationality

  243. The Failures of Eld Science

  244. The Dilemma: Science or Bayes?

  245. Science Doesn’t Trust Your Rationality

  246. When Science Can’t Help

  247. Science Isn’t Strict Enough

  248. Do Scientists Already Know This Stuff?

  249. No Safe Defense, Not Even Science

  250. Changing the Definition of Science

  251. Faster Than Science

  252. Einstein’s Speed

  253. That Alien Message

  254. My Childhood Role Model

  255. Einstein’s Superpowers

  256. Class Project

  Interlude: A Technical Explanation of Technical Explanation

  The World: An Introduction

  by Rob Bensinger

  Previous essays have discussed human reasoning, language, goals, and social dynamics. Mathematics, physics, and biology were cited to explain patterns in human behavior, but little has been said about humanity’s place in nature, or about the natural world in its own right.

  Just as it was useful to contrast humans as goal-oriented systems with inhuman processes in evolutionary biology and artificial intelligence, it will be useful in the coming sequences of essays to contrast humans as physical systems with inhuman processes that aren’t mind-like.

  We humans are, after all, built out of inhuman parts. The world of atoms looks nothing like the world as we ordinarily think of it, and certainly looks nothing like the world’s conscious denizens as we ordinarily think of them. As Giulio Giorello put the point in an interview with Daniel Dennett: “Yes, we have a soul. But it’s made of lots of tiny robots.”1

  Mere Reality collects seven sequences of essays on this topic. The first three introduce the question of how the human world relates to the world revealed by physics: “Lawful Truth” (on the basic links between physics and human cognition), “Reductionism 101” (on the project of scientifically explaining phenomena), and “Joy in the Merely Real” (on the emotional, personal significance of the scientific world-view). This is followed by two sequences that go into more depth on specific academic debates: “Physicalism 201” (on the hard problem of consciousness) and “Quantum Physics and Many Worlds” (on the measurement problem in physics). Finally, the sequence “Science and Rationality” and the essay A Technical Explanation of Technical Explanation tie these ideas together and relate them to scientific practice.

  The discussions of consciousness and quantum physics illustrate the relevance of reductionism to present-day controversies in science and philosophy. For those interested in a bit of extra context, I’ll say a few more words about those two topics here. For those eager to skip ahead: skip ahead!

  Minds in the World

  Can we ever know what it’s
like to be a bat?

  We can certainly develop better cognitive models for predicting bat behavior, or more fine-grained models of bat neurology—but it isn’t obvious that this would tell us what echolocation subjectively feels like, or what flying feels like, from the bat’s point of view.

  Indeed, it seems as though we could never even be certain that there is anything it’s like to be a bat. Why couldn’t an unconscious automaton replicate all the overt behaviors of a conscious agent to arbitrary precision? (Philosophers call such automata “zombies,” though they have little in common with the zombies of folklore—who are quite visibly different from conscious agents!)

  A race of alien psychologists would run into the same problem in trying to model human consciousness. They might arrive at a perfect predictive model of what we say and do when we see a red rose, but that wouldn’t mean that the aliens fully understand what redness feels like “from the inside.”

  Running with examples like these, philosophers like Thomas Nagel and David Chalmers have argued that third-person cognitive and neural models can never fully capture first-person consciousness.2,3 No matter how much we know about a physical system, it is always logically possible, on this view, that the system has no first-person experiences. Traditional dualism, with its immaterial souls freely floating around violating physical laws, may be false; but Chalmers insists on a weaker thesis, that consciousness is a “further fact” not fully explainable by the physical facts.

  A number of philosophers and scientists have found this line of reasoning persuasive.4 If we feel this argument’s intuitive force, should we grant its conclusion and ditch physicalism?

  We certainly shouldn’t reject it just because it sounds strange or feels vaguely unscientific. But how does the argument stand up to a technical understanding of how explanation and belief work? Are there any hints we can take from the history of science, or from our understanding of the physical mechanisms underlying evidence? “Physicalism 201” will return to this question.

  Worlds in the World

  Quantum mechanics is our best mathematical model of the universe to date, powerfully confirmed by a century of tests. The theory posits a complex-numbered “probability amplitude,” so called because a specific operation (squaring the number’s absolute value—the Born rule) lets us probabilistically predict phenomena at small scales and extreme energy levels. This amplitude changes deterministically in accord with the Schrödinger equation. In the process, it often enters odd states called “superpositions.”

  Yet when we perform experiments, the superpositions seem to vanish without a trace. When we aren’t looking, the Schrödinger equation appears to capture everything there is to know about the dynamics of physical systems. When we are looking, though, this clean determinism is replaced by Born’s probabilistic rule. It’s as though the ordinary laws of physics are suddenly suspended whenever we make “observations.” As John Stewart Bell put the point:

  It would seem that the theory is exclusively concerned about “results of measurements” and has nothing to say about anything else. What exactly qualifies some physical systems to play the role of the “measurer”? Was the wavefunction of the world waiting to jump for thousands of millions of years until a single-celled living creature appeared? Or did it have to wait a little longer, for some better qualified system . . . with a PhD?

  Everyone agrees that this strange mix of Schrödinger and Born’s rules has proved empirically adequate. However, the question of exactly when Born’s rule enters the mix, and what it all means, has produced a chaos of different views on the nature of quantum mechanics.

  Early on, the Copenhagen school—Niels Bohr and other originators of quantum theory—splintered into several standard ways of talking about the experimental results and the odd formalism used to predict them. Some, taking the theory’s focus on “measurements” and “observations” quite literally, proposed that consciousness plays a fundamental role in physical law, intervening to cause complex amplitudes to “collapse” into observables. Others, led by Werner Heisenberg, advocated a non-realistic view according to which physics is about our states of knowledge rather than about any objective reality. Yet another Copenhagen tradition, summed up in the slogan “shut up and calculate,” warned against metaphysical speculation of all kinds.

  Yudkowsky uses this scientific controversy as a proving ground for some central ideas from previous sequences: map-territory distinctions, mysterious answers, Bayesianism, and Occam’s Razor. Since he is not a physicist—and neither am I—I’ll provide some outside sources here for readers who want to vet his arguments or learn more about his physics examples.

  Tegmark’s Our Mathematical Universe discusses a number of relevant ideas in philosophy and physics.5 Among Tegmark’s more novel ideas is his argument that all consistent mathematical structures exist, including worlds with physical laws and boundary conditions entirely unlike our own. He distinguishes these Tegmark worlds from multiverses in more scientifically mainstream hypotheses—e.g., worlds in stochastic eternal inflationary models of the Big Bang and in Hugh Everett’s many-worlds interpretation of quantum physics.

  Yudkowsky discusses many-worlds interpretations at greater length, as a response to the Copenhagen interpretations of quantum mechanics. Many-worlds has become very popular in recent decades among physicists, especially cosmologists. However, a number of physicists continue to reject it or maintain agnosticism. For a (mostly) philosophically mainstream introduction to this debate, see Albert’s Quantum Mechanics and Experience.6 See also the Stanford Encyclopedia of Philosophy’s introduction to “Measurement in Quantum Theory,”7 and their introduction to several of the views associated with “many worlds” in “Everett’s Relative-State Formulation”8 and “Many-Worlds Interpretation.”9

  On the less theoretical side, Epstein’s Thinking Physics is a great text for training physical intuitions.10 It’s worth keeping in mind that just as one can understand most of cognitive science without understanding the nature of subjective awareness, one can understand most of physics without having a settled view of the ultimate nature (and size!) of the physical world.

  *

  1. Daniel C. Dennett, Freedom Evolves (Viking Books, 2003).

  2. David J. Chalmers, The Conscious Mind: In Search of a Fundamental Theory (New York: Oxford University Press, 1996).

  3. Thomas Nagel, “What Is It Like to Be a Bat?,” Philosophical Review 83, no. 4 (1974): 435–450, http://www.jstor.org/stable/2183914.

  4. In a survey of Anglophone professional philosophers, 56.5% endorsed physicalism, 27.1% endorsed anti-physicalism, and 16.4% endorsed other views (e.g., “I don’t know”).11 Most philosophers reject the metaphysical possibility of Chalmers’s “zombies,” but there is no consensus about why, exactly, Chalmers’s zombie argument fails. Kirk summarizes contemporary positions on phenomenal consciousness, giving arguments that resemble Yudkowsky’s against the possibility of knowing or referring to irreducible qualia.12

  5. Max Tegmark, Our Mathematical Universe: My Quest for the Ultimate Nature of Reality (Random House LLC, 2014).

  6. David Z. Albert, Quantum Mechanics and Experience (Harvard University Press, 1994).

  7. Henry Krips, “Measurement in Quantum Theory,” in The Stanford Encyclopedia of Philosophy, Fall 2013, ed. Edward N. Zalta.

  8. Jeffrey Barrett, Everett’s Relative-State Formulation of Quantum Mechanics, ed. Edward N. Zalta, http://plato.stanford.edu/archives/fall2008/entries/qm-everett/.

  9. Lev Vaidman, “Many-Worlds Interpretation of Quantum Mechanics,” in The Stanford Encyclopedia of Philosophy, Fall 2008, ed. Edward N. Zalta.

  10. Lewis Carroll Epstein, Thinking Physics: Understandable Practical Reality, 3rd Edition (Insight Press, 2009).

  11. David Bourget and David J. Chalmers, “What Do Philosophers Believe?,” Philosophical Studies (2013): 1–36.

  12. Robert Kirk, Mind and Body (McGill-Queen’s University Press, 2003).

  Part O


  Lawful Truth

  181

  Universal Fire

  In L. Sprague de Camp’s fantasy story The Incomplete Enchanter (which set the mold for the many imitations that followed), the hero, Harold Shea, is transported from our own universe into the universe of Norse mythology.1 This world is based on magic rather than technology; so naturally, when Our Hero tries to light a fire with a match brought along from Earth, the match fails to strike.

  I realize it was only a fantasy story, but . . . how do I put this . . .

  No.

  In the late eighteenth century, Antoine-Laurent de Lavoisier discovered fire. “What?” you say. “Hasn’t the use of fire been dated back for hundreds of thousands of years?” Well, yes, people used fire; it was hot, bright, sort of orangey-colored, and you could use it to cook things. But nobody knew how it worked. Greek and medieval alchemists thought that Fire was a basic thing, one of the Four Elements. In Lavoisier’s time the alchemical paradigm had been gradually amended and greatly complicated, but fire was still held to be basic—in the form of “phlogiston,” a rather mysterious substance which was said to explain fire, and also every other phenomenon in alchemy.

  Lavoisier’s great innovation was to weigh all the pieces of the chemical puzzle, both before and after the chemical reaction. It had previously been thought that some chemical transmutations changed the weight of the total material: If you subjected finely ground antimony to the focused sunlight of a burning glass, the antimony would be reduced to ashes after one hour, and the ashes would weigh one-tenth more than the original antimony—even though the burning had been accompanied by the loss of a thick white smoke. Lavoisier weighed all the components of such reactions, including the air in which the reaction took place, and discovered that matter was neither created nor destroyed. If the burnt ashes increased in weight, there was a corresponding decrease in the weight of the air.

 

‹ Prev