How Not to Be Wrong : The Power of Mathematical Thinking (9780698163843)

by Ellenberg, Jordan

  “It can hardly be supposed”: Charles Darwin, The Origin of Species, 6th ed. (London: 1872), 421.

  “the backbone of psychological research”: Richard J. Gerrig and Philip George Zimbardo, Psychology and Life (Boston: Allyn & Bacon, 2002).

  “only in his underwear”: David Bakan. “The Test of Significance in Psychological Research,” Psychological Bulletin 66, no. 6 (1966): 423–37.

  “New evidence has become available”: Quoted in Ann Furedi, “Social Consequences: The Public Health Implications of the 1995 ‘Pill Scare,’” Human Reproduction Update 5, no. 6 (1999): 621–26.

  “The government warned Thursday”: Edith M. Lederer, “Government Warns Some Birth Control Pills May Cause Blood Clots,” Associated Press, Oct. 19, 1995.

  stopped taking their contraceptives: Sally Hope, “Third Generation Oral Contraceptives: 12% of Women Stopped Taking Their Pill Immediately They Heard CSM’s Warning,” BMJ: British Medical Journal 312, no. 7030 (1996): 576.

  13,600 more abortions: Furedi, “Social Consequences,” 623.

  “possibly one”: Klim McPherson, “Third Generation Oral Contraception and Venous Thromboembolism,” BMJ: British Medical Journal 312, no. 7023 (1996): 68.

  A study by sociologists at CUNY: Julia Wrigley and Joanna Dreby, “Fatalities and the Organization of Child Care in the United States, 1985–2003,” American Sociological Review 70, no. 5 (2005): 729–57.

  who died of sudden infant death syndrome: All statistics about infant deaths are drawn from the Centers for Disease Control. Sherry L. Murphy, Jiaquan Xu, and Kenneth D. Kochanek, “Deaths: Final Data for 2010,” www.cdc.gov/nchs/data/nvsr/nvsr61/nvsr61/04.pdf.

  a frustrated novelist: Biographical material on Skinner is drawn from his autobiographical article “B. F. Skinner . . . An Autobiography” in Peter B. Dews, ed., Festschrift for BF Skinner (New York: Appleton-Century-Crofts, 1970), 1–22, and from his autobiography, Particulars of My Life, particularly pp. 262–63.

  “At night, he stops”: Skinner, “Autobiography,” 6.

  “a one-act play about a quack”: Ibid., 8.

  “All that makes a writer”: Skinner, Particulars, 262.

  “A violent reaction . . . at fault”: Skinner, “Autobiography,” 7.

  “Literature must be demolished”: Skinner, Particulars, 292.

  “No one has ever touched a soul”: John B. Watson, Behaviorism (Livingston, NJ: Transaction Publishers, 1998), 4.

  “You’ll never get out”: Skinner, “Autobiography,” 12.

  “emitting whole lines ready-made”: Skinner, “Autobiography,” 6.

  intentions of the author: Joshua Gang, “Behaviorism and the Beginnings of Close Reading,” ELH (English Literary History) 78, no. 1 (2011): pp. 1–25.

  “Proof that there is a process”: B. F. Skinner, “The Alliteration in Shakespeare’s Sonnets: A Study in Literary Behavior,” Psychological Record 3 (1939): 186–92. I learned about Skinner’s work on alliteration from Persi Diaconis and Frederick Mosteller’s classic paper “Methods for Studying Coincidences,” Journal of the American Statistical Association 84, no. 408 (1989), 853–61, essential reading if you want to go further into the ideas discussed in this chapter.

  “out of a hat”: Skinner, “Alliteration in Shakespeare’s Sonnets”, 191.

  As documented by literary historians: See, e.g., Ulrich K. Goldsmith, “Words out of a Hat? Alliteration and Assonance in Shakespeare’s Sonnets,” Journal of English and Germanic Philology 49, no. 1 (1950), 33–48.

  “Many writers indulge”: Herbert D. Ward, “The Trick of Alliteration,” North American Review 150, no. 398 (1890): 140–42.

  one of the most famous contemporary papers: Thomas Gilovich, Robert Vallone, and Amos Tversky, “The Hot Hand in Basketball: On the Misperception of Random Sequences,” Cognitive Psychology 17, no. 3 (1985): 295–314.

  Kevin Korb and Michael Stillwell: Kevin B. Korb and Michael Stillwell, “The Story of the Hot Hand: Powerful Myth or Powerless Critique?” (paper presented at the International Conference on Cognitive Science, 2003), www.csse.monash.edu.au/~korb/iccs.pdf.

  slightly more likely to make the next one: Gur Yaar and Shmuel Eisenmann, “The Hot (Invisible?) Hand: Can Time Sequence Patterns of Success/Failure in Sports Be Modeled as Repeated Random Independent Trials?” PLoS One, vol. 6, no. 10 (2011): e24532.

  vexed, murky, unsettled question: In this connection, I really like the 2011 paper “Differentiating Skill and Luck in Financial Markets with Streaks,” by Andrew Mauboussin and Samuel Arbesman—it’s an especially impressive piece of work considering the first author was a high school senior when it was written! I don’t think its conclusions are decisive, but I think it represents a very good way to think about these difficult problems. Available at papers.ssrn.com/sol3/papers.cfm?abstract_id=1664031.

  A 2009 study by John Huizinga and Sandy Weil: Personal communication with Huizinga.

  even more intriguing results along these lines: Yigal Attali, “Perceived Hotness Affects Behavior of Basketball Players and Coaches,” Psychological Science 24, no. 7 (July 1, 2013): 1151–56.

  Chapter 8: Reductio ad Unlikely

  there were only eighty-eight: Allison Klein, “Homicides Decrease in Washington Region,” Washington Post, Dec. 31, 2012.

  the study of the heavenly bodies: David W. Hughes and Susan Cartwright, “John Michell, the Pleiades, and Odds of 496,000 to 1,” Journal of Astronomical History and Heritage 10 (2007): 93–99.

  [figures]: The two pictures of points strewn across the square were generated by Yuval Peres of Microsoft Research and are taken from his “Gaussian Analytic Functions,” http://research.microsoft.com/en-us/um/people/peres/GAF/GAF.html.

  “The force with which such a conclusion”: Ronald A. Fisher, Statistical Methods and Scientific Inference (Edinburgh: Oliver & Boyd, 1959), 39.

  Joseph Berkson: Joseph Berkson, “Tests of Significance Considered as Evidence,” Journal of the American Statistical Association 37, no. 219 (1942), 325–35.

  In 2013, Yitang “Tom” Zhang: The story of Zhang’s work on the bounded gap conjecture is adapted from my article “The Beauty of Bounded Gaps,” Slate, May 22, 2013. See Yitang Zhang, “Bounded Gaps Between Primes,” Annals of Mathematics, forthcoming.

  Chapter 9: The International Journal of Haruspicy

  a parable I learned from the statistician Cosma Shalizi: You can read Shalizi’s own version at his blog, at http://bactra.org/weblog/698.html.

  John Ioannidis: John P. A. Ioannidis, “Why Most Published Research Findings Are False,” PLoS Medicine 2, no. 8 (2005): e124, available at www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124.

  Low power is a special danger: For an assessment of the perils of low-powered studies in neuroscience, see Katherine S. Button et al., “Power Failure: Why Small Sample Size Undermines the Reliability of Neuroscience,” Nature Reviews Neuroscience 14 (2013): 365–76.

  A recent paper in Psychological Science: Kristina M. Durante, Ashley Rae, and Vladas Griskevicius, “The Fluctuating Female Vote: Politics, Religion, and the Ovulatory Cycle,” Psychological Science 24, no. 6 (2013): 1007–16. I am grateful to Andrew Gelman for conversations about the methodology of this paper and for his blog post about it (http://andrewgelman.com/2013/05/17/how-can-statisticians-help-psychologists-do-their-research-better), from which my analysis is largely drawn.

  as likely to push you in the opposite direction: See Andrew Gelman and David Weakliem, “Of Beauty, Sex, and Power: Statistical Challenges in Estimating Small Effects,” American Scientist 97 (2009): 310–16, for a worked-out example of this phenomenon, in the context of the question of whether good-looking people have more daughters than sons. (Nope.)

  But when Chabris’s team tested those SNPs: Christopher F. Chabris et al., “Most Reported Genetic Associations with General Intelligence Are Probably False Positives,” Psychologic
al Science 23, no. 11 (2012): 1314–23.

  Amgen set out to replicate: C. Glenn Begley and Lee M. Ellis, “Drug Development: Raise Standards for Preclinical Cancer Research,” Nature 483, no. 7391 (2012): 531–33.

  calls these practices “p-hacking”: Uri Simonsohn, Leif Nelson, and Joseph Simmons, “P-Curve: A Key to the File Drawer,” Journal of Experimental Psychology: General, forthcoming. The curves sketched in this section are the “p-curves” described in this paper.

  political science to economics to psychology to sociology: Some representative references: Alan Gerber and Neil Malhotra, “Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals,” Quarterly Journal of Political Science 3, no. 3 (2008): 313–26; Alan S. Gerber and Neil Malhotra, “Publication Bias in Empirical Sociological Research: Do Arbitrary Significance Levels Distort Published Results?” Sociological Methods & Research 37, no. 1 (2008): 3–30; and E. J. Masicampo and Daniel R. Lalande, “A Peculiar Prevalence of P Values Just Below .05,” Quarterly Journal of Experimental Psychology 65, no. 11 (2012): 2271–79.

  the U.S. Supreme Court ruled unanimously: Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309, 563 U.S., 179 L. Ed. 2d 398 (2011).

  a paper by Robert Rector: Robert Rector and Kirk A. Johnson, “Adolescent Virginity Pledges and Risky Sexual Behaviors,” Heritage Foundation (2005), www.heritage.org/research/reports/2005/06/adolescent-virginity-pledges-and-risky-sexual-behaviors (accessed Jan. 14, 2014).

  “If a variable is not statistically significant”: Robert Rector, Kirk A. Johnson, and Patrick F. Fagan, “Understanding Differences in Black and White Child Poverty Rates,” Heritage Center for Data Analysis report CDA01-04 (2001), p. 15 (n. 20), quoted in Jordan Ellenberg, “Sex and Signifance,” Slate, July 5, 2005, http://thf_media.s3.amazonaws.com/2001/pdf/cda01-04.pdf (accessed Jan. 14, 2014).

  “the ordinary humanity of his fellows”: Michael Fitzgerald and Ioan James, The Mind of the Mathematician (Baltimore: Johns Hopkins University Press, 2007), 151, quoted in “The Widest Cleft in Statistics: How and Why Fisher Opposed Neyman and Pearson,” by Francisco Louçã, Department of Economics of the School of Economics and Management, Lisbon, Working Paper 02/2008/DE/UECE, available at www.iseg.utl.pt/departamentos/economia/wp/wp022008deuece.pdf (accessed Jan. 14, 2014). Note that the Fitzgerald-James book seems intent on arguing that a large number of successful mathematicians through history had Asperger’s syndrome, so their assessment of Fisher’s social development should be read with that in mind.

  “I am a little sorry”: Letter to Hick of Oct. 8, 1951, in J. H. Bennett, ed., Statistical Inference and Analysis: Selected Correspondence of R. A. Fisher (Oxford: Clarendon Press, 1990), 144. Quoted in Louçã, “Widest Cleft.”

  “A scientific fact should be regarded”: Ronald A. Fisher, “The Arrangement of Field Experiments,” Journal of the Ministry of Agriculture of Great Britain 33 (1926): 503–13, quoted in Jerry Dallal’s short article “Why p = 0.05?” (www.jerrydallal.com/LHSP/p05.htm)—a good introduction to Fisher’s thought on this issue.

  “in fact no scientific worker”: Ronald A. Fisher, Statistical Methods and Scientific Inference (Edinburgh: Oliver & Boyd, 1956), 41–42, also quoted in Dallal, “Why p = 0.05?”

  Chapter 10: Are You There, God? It’s Me, Bayesian Inference

  the Guest Marketing Analytics team at Target: Charles Duhigg, “How Companies Learn Your Secrets,” New York Times Magazine, Feb. 16, 2012.

  the computation was reproduced on a Nokia 6300: Peter Lynch and Owen Lynch, “Forecasts by PHONIAC,” Weather 63, no. 11 (2008): 324–26.

  a typical five-day forecast: Ian Roulstone and John Norbury, Invisible in the Storm: The Role of Mathematics in Understanding Weather (Princeton, NJ: Princeton University Press, 2013), 281.

  “One meteorologist remarked”: Edward N. Lorenz, “The Predictability of Hydrodynamic Flow,” Transactions of the New York Academy of Sciences, series 2, vol. 25, no. 4 (1963): 409–32.

  Lorenz thought it was about two weeks: Eugenia Kalnay, Atmospheric Modeling, Data Assimilation, and Predictability (Cambridge, UK: Cambridge University Press, 2003), 26.

  Netflix launched a $1 million competition: Jordan Ellenberg, “This Psychologist Might Outsmart the Math Brains Competing for the Netflix Prize,” Wired, Mar. 2008, pp. 114–22.

  which makes dud recommendations less of a big deal: Xavier Amatriain and Justin Basilico, “Netflix Recommendations: Beyond the 5 Stars,” techblog.netflix.com/2012/04/netflix-recommendations-beyond-5-stars.html (accessed Jan. 14, 2014).

  psychic powers were a hot topic: A good contemporary account of the ESP craze can be found in Francis Wickware, “Dr. Rhine and ESP,” Life, Apr. 15, 1940.

  were highly nonuniform: Thomas L. Griffiths and Joshua B. Tenenbaum, “Randomness and Coincidences: Reconciling Intuition and Probability Theory,” Proceedings of the 23rd Annual Conference of the Cognitive Science Society, 2001.

  17 is the most common choice: Personal communication, Gary Lupyan.

  they most frequently pick 7: Griffiths and Tenenbaum, “Randomness and Coincidences,” fig. 2.

  Two graduate students at Columbia: Bernd Beber and Alexandra Scacco, “The Devil Is in the Digits,” Washington Post, June 20, 2009.

  “If the views of the last section of Mr. Keynes’s book”: Ronald A. Fisher, “Mr. Keynes’s Treatise on Probability,” Eugenics Review 14, no.1 (1922): 46–50.

  “You know, the most amazing thing happened to me tonight”: Quoted by David Goodstein and Gerry Neugebauer in a special preface to the Feynman Lectures, reprinted in Richard Feynman, Six Easy Pieces (New York: Basic Books, 2011), xxi.

  The Cat in the Hat, the Cleanest Man in School, and the Creation of the Universe: The discussion in this section owes a great deal to Elliott Sober’s book Evidence and Evolution (New York: Cambridge University Press, 2008).

  fifteen editions in fifteen years: Aileen Fyfe, “The Reception of William Paley’s Natural Theology in the University of Cambridge,” British Journal for the History of Science 30, no. 106 (1997): 324.

  “I could almost formerly have said it by heart”: Letter from Darwin to John Lubbock, Nov. 22, 1859, Darwin Correspondence Project, www.darwinproject.ac.uk/letter/entry-2532 (accessed Jan. 14, 2014).

  Nick Bostrom: Nick Bostrom, “Are We Living in a Computer Simulation?” Philosophical Quarterly 53, no. 211 (2003): 243–55.

  a good argument that we’re all sims: Bostrom’s argument in favor of SIMS has more to it than this one; it’s controversial, but not immediately dismissible.

  Chapter 11: What to Expect When You’re Expecting to Win the Lottery

  seventeenth-century Genoa: All information on the Genoese lottery from David R. Bellhouse, “The Genoese Lottery,” Statistical Science 6, no. 2 (May 1991): 141–48.

  two new college buildings: Stoughton Hall and Holworthy Hall.

  “That the chance of gain is naturally overvalued”: Adam Smith, The Wealth of Nations (New York: Wiley, 2010), bk. 1, ch. 10, p. 102.

  the “Million Act” of 1692: The story of Halley and the mispriced annuity comes from chapter 13 of Ian Hacking, The Emergence of Probability (New York: Cambridge University Press, 1975).

  needed to be greater: See Edwin W. Kopf, “The Early History of the Annuity,” Proceedings of the Casualty Actuarial Society 13 (1926): 225–66.

  as many as 100 million tickets: personal communication from Powerball PR department.

  three times in 2012: “Jackpot History,” www.lottostrategies.com/script/jackpot_history/draw_date/101 (accessed Jan. 14, 2014).

  pick numbers other players won’t: See John Haigh, “The Statistics of Lotteries,” ch. 23 of Donald B. Hausch and William Thomas Ziemba, eds., Handbook of Sports and Lottery Markets (Amsterdam: Elsevier, 2008), for a survey of known results about which combinations lottery players prefer and which ones they shun.
<
br />   the exhaustive and, frankly, kind of thrilling account: Letter from Gregory W. Sullivan, Inspector General of the Commonwealth of Massachusetts, to Steven Grossman, State Treasurer of Massachusetts, July 27, 2012. The Sullivan report is the source for the material here about high-volume betting in Cash WinFall, except where otherwise specified; it’s available at www.mass.gov/ig/publications/reports-and-recommendations/2012/lottery-cash-winfall-letter-july-2012.pdf (accessed Jan. 14, 2014).

  They called their team Random Strategies: I couldn’t verify how early the name Random Strategies was actually chosen; it’s possible the team didn’t use this name when they were making their initial plays in 2005.

  an extra profit-making venture: Phone interview, Gerald Selbee, Feb. 11, 2013.

  “squarely within the square”: Thanks to François Dorais for this translation.

  a provincial aristocrat from Burgundy: The material on Buffon’s early life is drawn from chapters 1 and 2 of Jacques Roger, Buffon: A Life in Natural History, trans. Sarah Lucille Bonnefoi (Ithaca, NY: Cornell University Press, 1997).

  “But if instead of throwing”: From the translation of Buffon’s “Essay on Moral Arithmetic” by John D. Hey, Tibor M. Neugebauer, and Carmen M. Pasca, in Axel Ockenfels and Abdolkarim Sadrieh, The Selten School of Behavioral Economics (Berlin/Heidelberg: Springer-Verlag, 2010), 54.

  “Nothing seems to happen”: Pierre Deligne, “Quelques idées maîtresses de l’œuvre de A. Grothendieck,” in Matériaux pour l’histoire des mathématiques au XXe siècle: Actes du colloque à la mémoire de Jean Dieudonné, Nice, 1996 (Paris: Société Mathématique de France, 1998). The original is “rien ne semble de passer et pourtant à la fin de l’exposé un théorème clairement non trivial est là.” Translation by Colin McCarty, from his article “The Rising Sea: Grothendieck on Simplicity and Generality,” part 1, from Episodes in the History of Modern Algebra (1800–1950) (Providence: American Mathematical Society, 2007), 301–22.