The Signal and the Noise
Page 59
results in, 327
rich data in, 79–80, 84
Baseball America, 75, 87, 89, 90, 90, 91
Baseball Encyclopedia, 94
Baseball Prospectus, 75, 78, 88, 297
basic reproduction number (R0), 214–15, 215, 224, 225, 486
basketball, 80n, 92–93, 233–37, 243, 246, 256, 258, 489
batting average, 86, 91, 95, 100, 314, 321, 321, 339
Bayer Laboratories, 11–12, 249
Bayes, Thomas, 240–43, 251, 253, 254, 255, 490
Bayesian reasoning, 240, 241–42, 259, 349, 444
biases and beliefs in, 258–59
chess computers’ use of, 291
Christianity and, 490
in climatology, 371, 377–78, 403, 406–7, 407, 410–11
consensus opinion and, 367
Fisher’s opposition to, 252
gambling esteemed in, 255–56, 362
priors in, 244, 245, 246, 252, 255, 258–59, 260, 403, 406–7, 433n, 444, 451, 490, 497
stock market and, 259–60
Bayes’s theorem, 15, 16, 242, 243–49, 246, 247, 248, 249, 250, 258, 266, 331, 331, 448–49, 450–51
in poker, 299, 301, 304, 306, 307, 322–23
Beane, Billy, 77, 92, 93–94, 99–100, 103, 105–7, 314
Bear Stearns, 37
beauty, complexity and, 173
beer, 387, 459
behavioral economics, 227–28
Belgium, 459
Bellagio, 298–99, 300, 318, 495
bell-curve distribution, 368n, 496
Bengkulu, Indonesia, 161
Benjamin, Joel, 281
Berlin, Isaiah, 53
Berners-Lee, Tim, 448, 514
BetOnSports PLC, 319
bets, see gambling
Betsy, Hurricane, 140
betting markets, 201–3, 332–33
see also Intrade
biases, 12–13, 16, 293
Bayesian theory’s acknowledgment of, 258–59
in chess, 273
and errors in published research, 250
favorite-longshot, 497
of Fisher, 255
objectivity and, 72–73
toward overconfidence, 179–83, 191, 203, 454
in polls, 252–53
as rational, 197–99, 200
of scouts, 91–93, 102
of statheads, 91–93
of weather forecasts, 134–38
Bible, 2
Wicked, 3, 13
Biden, Joseph, 48
Big Data, 9–12, 197, 249–50, 253, 264, 289, 447, 452
Big Short, The (Lewis), 355
Billings, Darse, 324
Bill James Baseball Abstract, The, 77, 78, 84
bin Laden, Osama, 432, 433, 434, 440, 509
binomial distribution, 479
biological weapons, 437, 438, 443
biomedical research, 11–12, 183
bird flu, 209, 216, 229
Black, Fisher, 362, 367, 369
“Black Friday,” 320
Black Swan, The (Taleb), 368n
Black Tuesday, 349
Blanco, Kathleen, 140
Blankley, Tony, 50
Blodget, Henry, 352–54, 356, 364–65, 500
Blue Chip Economic Indicators survey, 199, 335–36
Bluefire, 110–11, 116, 118, 127, 131
bluffing, 301, 303, 306, 310, 311, 328
Bonus Baby rule, 94
books, 2–4
cost of producing, 2
forecasting and, 5
number of, 2–3, 3, 459
boom, dot-com, 346–48, 361
Boston, 77
Boston Red Sox, 63, 74–77, 87, 102, 103–5
Bowman, David, 161–62, 167
Box, George E. P., 230
Brady, Brian, 158, 159
brain, information storage capacity of, 12
breadth, depth vs., 271–73
breast cancer, 245–46, 246, 248, 250
Brier score, 474
British Medical Journal, 254
“broken windows” theory of crime, 439
Brown, Shawn, 226
Brownian noise, 173
Bryant, Kobe, 233–34, 237
bubbles, 38, 195, 333, 356, 357, 369–70
credit, 68, 196
difficulty of bursting, 360, 362, 367
dot-com, 346–48, 361
efficient-market hypothesis and, 346–52
housing, see housing bubble
real-time identification of, 347–48, 369–70
signals of, 366
Buchanan, Pat, 48, 50
bugs:
in Deep Blue, 283, 285, 286, 288–89
in models, 285–86
Bulgaria, 52
Bureau of Economic Analysis, 481
Bush, George H. W., 68
Bush, George W., 11, 55, 67, 443, 444, 510
business cycle, 185, 190
Business Insider, 353
Byrne, Donald, 286
Cain, Herman, 59
calibration, 134–36, 135, 136, 474
California, 22, 149–50, 427, 432
California Earthquake Prediction Council, 148
California Institute of Technology (Caltech), 146
Callendar, G. S., 401
caloric consumption, 372, 373
Cal State Fullerton, 161
Calvinism, 112
Campbell, Murray, 268, 284–85, 286, 288
Canada, 52, 210, 379
capitalism, 13
Protestant work ethic and, 5
see also free markets
CAPTCHA technology, 124
carbon dioxide, 374, 375, 379, 392–93, 395, 401–3, 404, 406, 408, 508
carbon emissions, 397–98, 399, 406, 410
Carew, Rod, 84, 85
Carley, Kathleen, 440
Carruthers, David, 319
Carter, Jimmy, 208
cartography, 3, 220
Case, Karl, 30, 32
Case-Shiller index, 30, 30, 464
causation, correlations vs., 185–88, 254–55
Center for Strategic and International Studies (CSIS), 425–27
Centers for Disease Control (CDC), 205, 206–7, 211
Central Park, 391, 391
CFOs, 359
Chadwick, Henry, 95
chaos, cone of, 139
chaos theory, 118–22, 124, 132, 162, 172, 195, 386
complexity theory vs., 386n
Charleston, S.C., 150
chartists, 339–40, 341
Chavez, Eric, 99
chemistry, 114
chess, 10, 16, 262–64, 263, 265–66, 493–94
beginning of game, 268–71
birth of computers for, 265–66
databases for, 270, 277
endgame of, 268, 276–79, 285
forecasting in, 271, 289
heuristics for, 267, 269, 272, 273, 284, 286
midgame of, 268, 271–76, 285
pattern detection in, 281
as theoretically solvable, 267
ChessBase.com, 292–93
Chicago, Ill., 223–24, 225, 228, 230, 432
Chicago, University of, 227
Chicago Cubs, 63, 104
Chicago White Sox, 88
Chile, 144, 438
China, 189, 209, 400
chlorofluorocarbons (CFCs), 375
Christchurch, New Zealand, 174
Christianity, 490
CIA, 51, 467
terrorism prevention by, 273, 424, 426, 433, 443, 510
CIBC Oppenheimer, 352
cigarette smoking, 254–55, 258
Cinema Rex theater, 425n
Cirque du Soleil, 318
Citizens Bank Park, 286
Civil Aeronautics Administration, 123n
Civil Protection Department, Italy, 143
Clarke, Richard, 425
Clauset, Aaron, 427, 431, 432, 437, 441, 442, 511–12
Clean Air Act, 400
Cleveland, Gr
over, 334
Cleveland Cavaliers, 239–40, 257
Clift, Eleanor, 48, 49, 50, 56
climate change, use of term, 376, 377n
see also global warming
Climategate, 408
Climatic Research Unit (CRU), 408
climatology, 131, 132, 370–411
Bayesian reasoning in, 371, 377–78, 403, 406–7, 407, 410–11
models of, 371, 380, 384–85, 401–6, 402
signal vs. noise in, 371–73
uncertainty in, 389–93, 390
Clinton, Bill, 55, 56, 433, 510
Clinton, Hillary, 59, 60, 252
clouds, 385, 386
CNN, 217
coal, 410
cognitive psychology, 227
Cole, USS, 422, 423
comets, 447
commerce, 10
Commerce Department, U.S., 123n
commercial lending, 187
commodity prices, 186n, 202
common sense, 451
communism, 51
community cards, 299
compartments, in disease modeling, 220–21, 223
competition, 1, 16, 97, 106, 128, 189
in poker, 313
in the stock market, 313, 352, 364
in weather forecasting, 127–28, 131–37, 132
competitive advantage, 313–14
competitiveness, 97
complexity, of global warming forecasting, 382
complexity theory, 172–73, 368–69, 386
chaos theory vs., 386ncomputer age, 7–8
computers:
chess played by, 261–62, 287–88; see also Deep Blue; Deep Thought; Fritz
poker played by, 324
predictions and, 292
weather forecasting by, 116–18, 123–25, 289
condom fatigue, 222–23
cone of chaos, 139
Conference Board, 187
confidence, 46
accuracy and, 203
see also overconfidence
confidence interval, see margin of error
Congress, U.S., 19, 123n, 207, 408
low approval rating of, 188
see also House of Representatives, U.S.; Senate, U.S.
Connecticut, 391
consensus, 66–67, 331–32, 335–36, 367, 451
of global-warming forecasting, 382–84
conservatism, 509
conspiracy theories, 417n
consumer confidence, 186n, 187–88
consumer spending, 186n, 187
context:
as ignored in failed predictions, 43
statistics and, 79, 84, 91, 100–102, 105, 234, 240
contrarianism, 380
Cook, Charlie, 69–70
Cook County Jail, 228
Cook Political Report, 69–72, 100n, 469
Copenhagen climate conference, 378–80
correlations, causation vs., 185–88, 254–55
Council of Economic Advisers, 40
Council on Foreign Relations, 435
Cramton, Steven, 292–93
creativity, 287–88, 289, 290, 291, 311
credit bubble, 68, 196
credit default option (CDO), 20–21, 21, 22, 24, 25, 26–30, 36, 43, 462
tranches of, 26–28, 28
credit default swap, 36
credit markets, 19
Crist, Charlie, 33
Cronkite, Walter, 207–8
Crowley, Monica, 48–49, 467
crystal methamphetamine, 221
Cuban Missile Crisis, 419, 433
Curb Your Enthusiasm, 111
cyclones, see hurricanes
Czechoslovakia, 52
Daily Kos, 60
Damon, Matt, 317
Dark Winter, 437
Darwin, Charles, 375
data:
in baseball, 79–80, 84
Big, 9–12, 197, 250, 253, 264, 289, 447, 452
distribution of, 164, 165
in economics, 80, 185, 193–94
in frequentism, 253
overfitting of, 163–71, 166, 168–71, 185, 191, 452n, 478
Pareto principle and, 313
data mining, 298
Daum, Robert, 224, 227, 229
David, Larry, 111
Davis, Ricky, 239, 257
De Bernardinis, Bernardo, 143
debt crisis, European, 198
Deep Blue, 10, 266, 268, 292, 493–94
bug in, 283, 285, 286, 288–89
creation of, 283–85
Kasparov’s final games against, 282–83
Kasparov’s first game against, 268, 270–79, 271, 274, 275, 276, 278
Kasparov’s second game against, 279–82, 280
rook moved for no apparent purpose by, 277–79, 278, 288
Deep Thought, 268, 284
default, 20–21, 22, 27–29, 184
defense, 90, 92, 106
Defense Department, U.S., terrorism prevention by, 273
defensive range, 96
de Groot, Adriaan, 272
Denver, Colo., 150
Denver Post, 176
depth, breadth vs., 271–73
determinism, 112, 113, 241, 242, 249, 448
Detroit Tigers, 77, 88, 94
difference engine, 263
Discover, 160
discrimination, calibration vs., 474
disease, see infectious disease
diversification, 27
“Divine Benevolence” (Bayes), 241, 242
Djokovic, Novak, 496
Dodger Stadium, 79
Dokhoian, Yuri, 282
Domodedovo Airport, 440
dot-com boom, 346–48, 361
Dow Jones Industrial Average, 37, 339, 340, 343, 498, 503
Doyle, Arthur Conan, 307
Drake equation, 488
Druckenmiller, Stanley, 356
Dukakis, Michael, 68
Duke University, 359
Dutch book, 256n
Dwan, Tom, 308–11, 313, 315, 318, 324, 328
dynamic systems, 16, 118, 120, 194
E*Trade, 339, 363, 498
earthquake forecasting, 149–54, 230
computers in, 289
failure of, 7, 11, 143, 147–49, 158–61, 168–71, 174, 249, 253, 346, 389
overfitting and, 168–71, 185
short-term, 174
time-dependent, 154
earthquakes, 16, 142–75, 476, 512
aftershocks to, 154, 161, 174, 476–77
in Anchorage, 149
causes of, 162
distribution across time and space of, 154–57, 155, 427
foreshocks to, 144, 154, 155–57, 476
Great Sumatra, 161, 171
in Haiti, 147n, 155–56, 156, 224
in Japan, 154, 155, 168–71, 172
in L’Aquila, 142–44, 148, 154–55, 157, 173
Lisbon, 145
list of deadliest, 147
Loma Prieta, 160
magnitude vs. frequency of, 151–53, 152, 153, 368n, 427, 432, 437–38, 441
near Reno, 156–57, 157
in New Zealand, 174
earthquake swarm, 143n, 173
Earth System Science Center, 408
East Germany, 52
eBay, 353
Ecclesiastes, 459
economic data, noise in, 193–94, 198
economic growth, 6, 6, 186n
economic progress, 7, 112, 243
economics, predictions in, 33, 176–77, 230
actual GDP vs., 191–93, 192, 193, 194
Big Data and, 197
computers in, 289
consensus vs. individual, 197–98, 335
context ignored in, 43
an ever-changing economy, 189–93
economics, predictions in (Cont.)
failures of, 11, 14, 40–42, 41, 45, 53, 162, 179–88, 182, 198, 200–201, 249, 388, 477, 479
feedback and, 188, 195
“leading indicators” in, 186–88, 196–97<
br />
overconfidence in, 179–83, 191, 203
risk and, 203
three challenges of, 185
economics, small data in, 80, 185, 193–94
economic value, of forecasts, 129–30
Economist, The, 22, 434–35
ECRI, 196–97
efficient-market hypothesis, 333, 337–38, 449–50
bubbles and, 360
complications of, 356
irrational exuberance vs., 346–52
as self-defeating, 363
test of, 343–46
three forms of, 341–43
two components in, 361–62
effort, predictive success vs., 312
Eggleston, J. D., 136–37
eHarmony, 426
Ehrlich, Anne, 212–13
Ehrlich, Paul R., 212–13
Eiffel Tower, 422
80–20 rule, 312–13, 315, 316n, 317
Eighty Years’ War, 4
Eisenhower, Dwight D., 67
Eisenstein, Elizabeth, 4
elections, U.S.:
of 1892, 334
of 1988, 67
of 2000, 11, 67, 68, 468–69
of 2008, 19, 48–49, 55, 56, 59, 60, 252, 426, 468
of 2010, 57–58, 61, 64
of 2012, 59, 65, 333, 336
electrical engineering, 173
Elements of Poker (Angelo), 325
Elias Baseball Analyst, 77
Ellsbury, Jacoby, 101
El Niño-Southern Oscillation (ENSO) cycle, 392, 393, 401, 403
Emanuel, Kerry, 384–85, 387
Emanuel, Rahm, 48
English Civil War, 4
Enlightenment, 2, 112
Enron, 356
entitlement, sense of, 326
epidemiology, 204–31
basic reproduction number in, 214–15, 215, 224, 225
extrapolation in, 212–16
self-canceling predictions in, 219–20
self-fulfilling predictions in, 217–18
SIR model in, 220–21, 221, 223, 225, 389
equity holdings, 357
equity premium puzzle, 349n
ERA, 91, 95
ESPN, 294–95, 308
“Essay Toward Solving a Problem in the Doctrine of Chances,” 241
Estonia, 50–51, 52
ethnography, 228
European debt crisis, 198
European Union, 397–98
evacuations, 126–27
evolution, 292
EVTMX (Eaton Vance Dividend Builder A), 339–40
experience, 57, 312
Expert Political Judgment (Tetlock), 52
experts, 14, 52–53
definition of, 467
demand for, 202
extrapolation, 212–16
eyesight, 123, 124
FAA, 423
Fair, Ray C., 482
false negatives, 372
false positives, 245, 249–50, 251, 253
falsifiability, 14–15, 452
Fama, Eugene, 337–38, 339–40, 341, 346, 347, 353, 362, 363, 368n, 497
Fannie Mae, 33
fashion, 217
fatalism, 5
fate:
American belief in control of, 10
in Julius Caesar, 4–5, 10
fat tails, 368, 496
fault lines, 162
favorite-longshot bias, 497