The Signal and the Noise

by Nate Silver

63. Although there are several different formulations of the steps in the scientific method, this version is mostly drawn from “APPENDIX E: Introduction to the Scientific Method,” University of Rochester. http://teacher.pas.rochester.edu/phy_labs/appendixe/appendixe.html.

  64. Thomas S. Kuhn, The Structure of Scientific Revolutions (Chicago: University of Chicago Press, Kindle edition).

  65. Jacob Cohen, “The Earth Is Round (p<.05),” American Psychologist, 49, 12 (December 1994), pp. 997–1003. http://ist-socrates.berkeley.edu/~maccoun/PP279_Cohen1.pdf.

  66. Jeff Gill, “The Insignificance of Null Hypothesis Significance Testing,” Political Research Quarterly, 52, 3 (September 1999), pp. 647–674. http://www.artsci.wustl.edu/~jgill/papers/hypo.pdf.

  67. David R. Anderson, Kenneth P. Burnham, and William L. Thompson, “Null Hypothesis Testing: Problems, Prevalence, and an Alternative,” Journal of Wildlife Management, 64, 4 (2000), pp. 912–923. http://cat.inist.fr/%3FaModele%3DafficheN%26cpsidt%3D792848.

  68. William M. Briggs, “It Is Time to Stop Teaching Frequentism to Non-Statisticians,” arXiv.org, January 13, 2012. http://arxiv.org/pdf/1201.2590.pdf.

  69. David H. Krantz, “The Null Hypothesis Testing Controversy in Psychology,” Journal of the American Statistical Association, 44, no. 448 (December 1999). http://www.jstor.org/discover/10.2307/2669949?uid=3739832&uid=2&uid=4&uid=3739256&sid=47698905120317.

  CHAPTER 9. RAGE AGAINST THE MACHINES

  1. “Poe Invents the Modern Detective Story,” National Historic Site Philadelphia, National Park Service, U.S. Department of the Interior. http://www.nps.gov/edal/forteachers/upload/detective.pdf.

  2. Nick Eaton, “Gallup: Bill Gates Is America’s Fifth-Most Admired Man,” Seattle Post-Intelligencer, December 27, 2010. http://blog.seattlepi.com/microsoft/2010/12/27/gallup-bill-gates-is-americas-fifth-most-admired-man/.

  3. Joann Pan, “Apple Tops Fortune’s ‘Most Admired’ List for Fifth Year in a Row,” Mashable, March 2, 2012. http://mashable.com/2012/03/02/apple-tops-fortunes-most-admired-list-five-years-straight-video/.

  4. David Kravets, “Stock-Picking Robot ‘Marl’ Is a Fraud, SEC Says,” Threat Level, Wired, April 23, 2012. http://www.wired.com/threatlevel/2012/04/stock-picking-robot/.

  5. “What Is the Stock Trading Robot ‘MARL’?,” Squidoo.com. http://www.squidoo.com/StockTradingRobotMARL.

  6. Philadelphia Inquirer, “Computer Predicts Odds of Life, Death,” Orlando Sentinel, July 9, 1992. http://articles.orlandosentinel.com/1992-07-09/news/9207090066_1_apache-system-critical-care-critical-care.

  7. Nick Montfort, Twisty Little Passages: An Approach to Interactive Fiction (Boston: MIT Press, 2005), p. 76.

  8. Claude E. Shannon, “Programming a Computer for Playing Chess,” Philosophical Magazine, Series 7, 41, 314, March 1950. http://archive.computerhistory.org/projects/ chess/related_materials/software/2-0%20and%202-1.Programming_a_computer_for_playing_chess.shannon/2-0%20and%202-1.Programming_a_computer_for_playing_chess.shannon.062303002.pdf.

  9. William G. Chase and Herbert A. Simon, “The Mind’s Eye in Chess” in Visual Information Processing (New York: Academic Press, 1973).

  10. Douglas Harper, Online Etymology Dictionary. http://www.etymonline.com/index.php?term=eureka.

  11. Amos Tversky and Daniel Kahneman, “Judgement Under Uncertainty: Heuristics and Biases,” Science, 185 (September 27, 1974), pp. 1124–1131. http://www.econ.yale.edu/~nordhaus/homepage/documents/tversky_kahn_science.pdf.

  12. Lauren Himiak, “Bear Safety Tips,” National & States Parks, About.com. http://usparks.about.com/od/backcountry/a/Bear-Safety.htm.

  13. billwall, “Who Is the Strongest Chess Player?” Chess.com, October 27, 2008. http://www.chess.com/article/view/who-is-the-strongest-chess-player.

  14. Feng-hsiung Hsu, Thomas Anantharaman, Murray Campbell, and Andreas Nowatzyk, “A Grandmaster Chess Machine,” Scientific American, October 1990. http://www.disi.unige.it/person/DelzannoG/AI2/hsu.html.

  15. Ibid.

  16. “The Chip vs. the Chessmaster,” Nova (documentary), March 26, 1991.

  17. Garry Kasparov, “The Chess Master and the Computer,” New York Review of Books, February 11, 2010. http://www.nybooks.com/articles/archives/2010/feb/11/the-chess-master-and-the-computer/.

  18. “Frequently Asked Questions: Deep Blue;” IBM Research via Internet Archive WayBack Machine beta. http://web.archive.org/web/20071028124110/http://www.research.ibm.com/deepblue/meet/html/d.3.3a.shtml#difficult.

  19. Chess Opening Explorer, chessgames.com. http://www.chessgames.com/perl/explorer.

  20. Murray Campbell, A. Joseph Hoane Jr., and Feng-hsiung Hsu, “Deep Blue,” sjeng.org, August 1, 2001. http://sjeng.org/ftp/deepblue.pdf.

  21. IBM Research, “Frequently Asked Questions: Deep Blue.”

  22. “1, Nf3 d5, 2. g3 Bg4” Chess Opening Explorer, chessgames.com. http://www.chessgames.com/perl/explorer?node=1959282&move=3&moves=Nf3.d5.g3.Bg4&nodes=74.77705.124843.1959282.

  23. Trading a bishop for a knight early in the game, as Deep Blue threatened to do, may not be a good trade because bishops are more valuable when a player still has both of them on the board. If you have just one bishop, your opponent can move with relative impunity on the half of the squares that the remaining bishop can’t physically cover. In other words, you’d rather have one knight and both bishops than two knights and one bishop.

  24. Position Search, chessgames.com. http://www.chessgames.com/perl/chess.pl?node=1967201.

  25. Adriaan D. de Groot, Thought and Choice in Chess (Amsterdam, Holland: Amsterdam University Press, Amsterdam Academic Archive, 2008).

  26. Ibid.

  27. Shannon, “Programming a Computer for Playing Chess.”

  28. Uly, January 23, 2010 (2:52 P.M.), comment on “computer eval – winning chances” by ppipper, on Rybka Chess Community Forum. http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=15144.

  29. “Kasparov vs. Deep Blue, Game 1, May 3, 1997,” Chess Corner. http://www.chesscorner.com/games/deepblue/dblue1.htm.

  30. Robert Byrne, “In Late Flourish, a Human Outcalculates a Calculator,” New York Times, May 4, 1997. http://www.nytimes.com/1997/05/04/nyregion/in-late-flourish-a-human-outcalculates-a-calculator.html?scp=3&sq=kasparov&st=nyt.

  31. deka, “Analysis by Rybka 3 14ply,” February 26, 2010. http://web.zone.ee/chessanalysis/study%20on%20chess%20strength.pdf.

  32. Frederic Friedel, “Garry Kasparov vs. Deep Blue,” ChessBase.com, May 1997. http://www.chessbase.com/columns/column.asp?pid=146.

  33. Ibid.

  34. Ibid.

  35. “Deep Blue: Overview,” IBM100 Icons of Progress, IBM. http://www.research.ibm.com/deepblue/games/game2/html/move34b.shtml.

  36. Which was the correct move is still debated. The other computers of the day—less advanced than Deep Blue—found Qb6, the queen advancement, to be the right play, and by a somewhat large margin. But the very fact that Deep Blue had deviated from the computer play had been what distinguished its move. When I set up the position on a contemporary computer engine called Rybka, it also picked Qb6, but found the position to be much closer, the difference amounting to not more than about three-tenths of a pawn worth of strength. The difference is small enough that it’s easy to imagine Deep Blue, a somewhat idiosyncratic engine that was specifically tailored to match up with Kasparov, picking the alternative line.

  37. Maurice Ashley, Patrick Wolff, and Yasser Seirawan, “Game 2, black 36 . . . axb5,” IBM Research real-time text commentary, May 11, 2007. http://web.archive.org/web/20080614011112/http://www.research.ibm.com/deepblue/games/game2/html/move36b.shtml.

  38. The actual match itself was played upstairs, in a specially designed television studio on the thirty-fifth floor of the Equitable Center, with no spectators permitted.

  39. Bruce Weber, “Computer Defeats Kasparov, Stunning the Chess Experts,” New York Times, May 5, 1997. http://www.nytimes.com/1997/05/05/nyregion/computer-defeats-kasparov-stunning-the-chess-experts.html?scp=3&sq=kasparov&st=nyt
.

  40. Bruce Weber, “Wary Kasparov and Deep Blue Draw Game 3,” New York Times, May 7, 1997. http://www.nytimes.com/1997/05/07/nyregion/wary-kasparov-and-deep-blue-draw-game-3.html?scp=1&sq=kasparov+hand+of+god&st=nyt.

  41. Frederic Friedel, “Garry Kasparov vs. Deep Blue,” Multimedia Report, ChessBase Magazine 58. http://www.chessbase.com/columns/column.asp?pid=146.

  42. Bruce Weber, “Swift and Slashing, Computer Topples Kasparov,” New York Times, May 12, 1997. http://www.nytimes.com/1997/05/12/nyregion/swift-and-slashing-computer-topples-kasparov.html?scp=3&sq=kasparov&st=nyt.

  43. This metaphor is borrowed from Bill Wyman, a music critic for the Chicago Reader, who ranked it as the greatest moment in rock history. Bill Wyman, “The 100 Greatest Moments in Rock History,” Chicago Reader, September 28, 1995. http://www.chicagoreader.com/chicago/the-100-greatest-moments-in-rock-history/Content?oid=888578.

  44. Campbell, Hoane Jr., and Feng-hsiung, “Deep Blue.”

  45. Larry Page, “PageRank: Bringing Order to the Web,” Stanford Digital Library Project, August 18, 1997. http://web.archive.org/web/20020506051802/www-diglib.stanford.edu/cgi-bin/WP/get/SIDL-WP-1997-0072?1.

  46. “How Search Works,” by Google via YouTube, March 4, 2010. http://www.youtube.com/watch?v=BNHR6IQJGZs.

  47. Per interview with Vasik Rajlich.

  48. “Amateurs beat GMs in PAL / CSS Freestyle,” ChessBase News. http://www.chessbase.com/newsdetail.asp?newsid=2467.

  49. Kasparov, “The Chess Master and the Computer.”

  CHAPTER 10. THE POKER BUBBLE

  1. “Chris Moneymaker Ranking History” in The Mob Poker Database, thehendonmob.com. http://pokerdb.thehendonmob.com/player_graphs/chris_moneymaker_18826.

  2. I first played in one of the smaller events at the World Series of Poker in 2005, although I did not participate in the $10,000 main event of the tournament until 2009.

  3. The catch was that you had to play a certain number of hands at the site before you could cash out any winnings.

  4. The example that follows is idealized, in the sense that we will be applying a relatively formal and rigorous mathematical process to consider a fairly typical poker hand, alongside the more impressionistic view that a player might have at the table. In a real poker game, players must make decisions much more quickly, both for reasons of etiquette and because spending too much time contemplating a hand would itself reveal information to the opponents. However, the thought process described here is what all poker players are striving for, whether they realize it or not. The question is who can come to the closest approximation of it under the pressures that a real game represents.

  5. If the opponent raised with a middling hand, such as a pair of nines on the river, he would essentially be doing so as a bluff since he wouldn’t expect us to call with a weaker hand.

  6. Nate Silver, “Sanity Check: 88 Hand” twoplustwo.com; May 14, 2012. http://forumserver.twoplustwo.com/56/medium-stakes-pl-nl/sanity-check-88-hand-1199549/.

  7. G4mblers, “Biggest Pot in TV Poker History—Tom Dwan vs Phil Ivey Over 1.1 Million,” YouTube; January 28, 2010. http://www.youtube.com/watch?v=GnxFohpljqM.

  8. “About Tom Dwan;” PokerListings.com. http://www.pokerlistings.com/poker-player_tom-dwan.

  9. locke, “Isildur1 & the Poker Economy,” PokerTableRatings.com, December 11, 2009. http://www.pokertableratings.com/blog/2009/12/isildur1-the-poker-economy/.

  10. “Player Profile: durrrr;” Highstakes DataBase. http://www.highstakesdb.com/profiles/durrrr.aspx.

  11. PokerListings.com, “About Tom Dwan.”

  12. The quotes included in this book are from an interview I conducted by phone with Dwan in May 2012, but I had met him in person on previous occasions.

  13. Bill Chen and Jerrod Ankenman, “The Mathematics of Poker,” Conjelco, November 30, 2006.

  14. Darse Billings, “Algorithms and Assessment in Computer Poker,” thesis submitted to Department of Computing Science, University of Alberta; 2006. http://www.cs.virginia.edu/~evans/poker/readings/billings-ch1.pdf.

  15. Tommy Angelo, “Elements of Poker,” Tommy Angelo Operations, Kindle edition, p. 209, December 13, 2007.

  16. Robert Koch, Living Life the 80/20 Way (Boston: Nicholas Brealey Publishing, 2004).

  17. My analysis was limited to players who played in games featuring a big blind of $2 or higher; this is about the cheapest increment at which enough money could potentially be made from the game to sustain the existence of some professional players.

  18. Specifically, I split the sample data into even- and odd-numbered months; if a player is truly skilled, he should be winning in the even-numbered months as well as the odd-numbered ones. I then applied a regression analysis to predict a player’s win rate (measured as big blinds won per one hundred hands) from one half of the sample to the other; the results of the regression are taken to be tantamount to the player’s long term success rate. The variables in the regression were a player’s win rate, multiplied by the natural logarithm of the number of hands who he played, along with a variable indicating how many hands he played of the ones he was dealt. Players who were too tight or too loose were considerably less likely to replicate their success from one period to the next, holding other factors equal—in fact, this was often a better predictor of a player’s out-of-sample win rate than the past win rate itself, unless he had played a very large number of hands.

  19. Online poker tables typically consisted of ten seats; those in bricks-and-mortar casinos often contain nine seats instead.

  20. In the online poker data, the rake was equivalent to about $57 per one hundred hands, according to my estimate. The figure would potentially be quite similar at a bricks-and-mortar casino. The Bellagio, for instance, usually charges players a fee of $6 per half hour. To get in one hundred hands of no-limit poker in a casino environment, a player would typically need to spend about four hours at the table (it is a fairly slow game). That means she’d need to pay the time charge eight times, for $48 total. Add in the customary $1 tip to the dealer when she wins a hand, and you get to roughly the same $57 figure. No adjustment was made for the benefits—and the hidden costs—of the free cocktails in Las Vegas.

  21. “Listen, here’s the thing, if you can’t spot the sucker in your first half hour at the table, then you are the sucker,” Mike McDermott in Rounders via monologuedb.com. http://www.monologuedb.com/dramatic-male-monologues/rounders-mike-mcdermott/.

  22. “Guy Laliberte’s Accounts on Full Tilt Poker Down Millions of Dollars in 2008,” PokerKingBlog.com, January 2, 2009. http://www.pokerkingblog.com/2009/01/02/guy-laliberte-the-engine-of-the-high-stakes-economy-on-full-tilt-poker/.

  23. Games tended to be a bit fishier in the winter, when more people were staying indoors and were logged on to their computers, and in the summer when the World Series of Poker was being played, than they were in the spring or the fall.

  24. James McManus, “Full Tilt Boogie: The UIGEA and You,” Grantland.com, December 8, 2011. http://www.grantland.com/story/_/id/7333093/uigea-you.

  25. Rocco Havel, “Taking Stock of the UIGEA,” Tight Poker, April 16, 2008. http://www.tightpoker.com/news/taking-stock-of-the-uigea-487/.

  26. Branon Adams, “The Poker Economy,” Bluff Magazine, November, 2006. http://www.bluffmagazine.com/magazine/The-Poker-Economy-Brandon-Adams-584.htm.

  27. Nate Silver, “After ‘Black Friday,’ American Poker Faces Cloudy Future,” FiveThirtyEight, New York Times, April 20, 2011. http://fivethirtyeight.blogs.nytimes.com/2011/04/20/after-black-friday-american-poker-faces-cloudy-future/.

  28. Based on a minimum of fifty at-bats for the player in both April and May.

  29. Contrast this with a sport like tennis, where the structure of the game is such that even modest differences in skill level manifest themselves very quickly. The very best players in the world, like Rafael Nadal and Novak Djokovic, win only about 55 percent of the points they play—barely more than half. However, hundreds of points are played over the course
of a single match, while baseball players might get just four or five at-bats. It would be silly to conclude that Nadal is better at tennis than Josh Hamilton is at baseball. But Nadal almost always wins while Hamilton has plenty of nights when he goes 0-for-4. In tennis, you reach the long run much more quickly.

  30. This estimate is based on my own history, as well as the statistics from other limit hold ’em players in my databases.

  31. Chen and Ankenman, “The Mathematics of Poker;” pp. 40–43.

  32. Although this calculation is made more complicated by the fact that the distribution of long-term win rates among poker players almost certainly does not follow a bell curve (normal distribution). Instead, per our application of the Pareto principle of prediction, it is left-skewed, with a “fat tail” of losing players.

  33. Martin Harris, “Polaris 2.0 Defeats Stoxpoker Team in Man-Machine Poker Championship Rematch,” PokerNews.com, July 10, 2008. http://www.pokernews.com/news/2008/07/man-machine-II-poker-championship-polaris-defeats-stoxpoker-.htm.

  34. “Poker Services;” Poker Royalty. http://pokerroyalty.com/poker-business.php.

  35. “Annual per Capita Lottery Sales, by Educational Attainment,” bp0.blogger.com. http://bp0.blogger.com/_bYktpmgngXA/RclJid4kTxI/AAAAAAAAAHU/PnDE3-Orpqc/s1600-h/Compound_Charts3.JPG.

  36. Angelo, “Elements of Poker,” Kindle location 2.

  37. Ibid., Kindle location 55.

  CHAPTER 11. IF YOU CAN’T BEAT ’EM . . .

  1. “Stocks Traded, Total Value (% of GDP),” World Bank World Development Indicators. http://data.worldbank.org/indicator/CM.MKT.TRAD.GD.ZS.

  2. “Fortune 500;” CNN Money. http://money.cnn.com/magazines/fortune/fortune500/2009/full_list/101_200.html.

  3. “Stocks Traded, Turnover Ratio (%),”World Bank World Development Indicators. http://data.worldbank.org/indicator/CM.MKT.TRNR/countries.

  4. Adrianne Jeffries, “High-Frequency Trading Approaches the Speed of Light,” BetaBeat.com, February 17, 2012. http://www.betabeat.com/2012/02/17/high-frequency-trading-approaches-the-speed-of-light/.

  5. Terrance Odean, “Do Investors Trade Too Much?” American Economic Review, 89, no. 5 (December 1999), pp. 1279–1298. http://web.ku.edu/~finpko/myssi/FIN938/Odean_Do%20Investors%20Trade%20Too%20Much_AER_1999.pdf.