2 R. Winkler, “Evaluating Probabilities: Asymmetric Scoring Rules,” Management Science 40 (1994): 1395–1405.
3 A. Tversky and D. Kahneman, “Advances in Prospect Theory: Cumulative Representation of Uncertainty,” Journal of Risk and Uncertainty 5 (1992): 297–323; A. Tversky and C. R. Fox, “Weighing Risk and Uncertainty,” Psychological Review 102 (1995): 269–83.
4 C. Ragin, Fuzzy-set Social Science (Chicago: University of Chicago Press, 2000).
5 A. Tversky and D. J. Koehler, “Support Theory: A Nonextensional Representation of Subjective Probability,” Psychological Review 101 (1994): 547–67.
Index
Abelson, Robert, 39n. 36, 182–83n. 16
academic hyperspecialization, 233
academic journals, 233–34
accountability, 185–86, 218n. 2
accuracy criteria, 249–50; for domestic political leadership, 249; for government policy and economic performance, 249–50; for national security, 250
Acheson, Dean, 143
actor-dispensability debates, 106–7
additive rule, 301; violations of, 303–5. See also sub-additivity; support theory; “unpacking” scenarios
African National Congress (ANC), 109–10
AIDS, 225
Allen, P. G., 42n. 42
Allison, G., xiii n. 4
Almond, G., 134n. 11
Al Qaeda, 6
ambiguity, 38; aversion to ambiguity as factor driving fox-hedgehog performance differentials, 81–82
American Political Science Association, 25
analogical reasoning (from history), 38, 92–100
Anderson, C., 191n. 3
Angell, Norman, 102
antideterminists, 153
Argentina, 114, 115
Arkes, H., 65n. 50, 123n. 2
Armstrong, J. S., 65n. 50, 86n. 16, 1 18n. 46
Arthur, B., 27n. 8
Articles of Confederation, 89
“Asian Tigers,” 116
Bartels, L., 25n. 1
base rates, 42n. 43, 49, 220; prediction strategies (using contemporaneous or recent past cross-sectional base rates, defined either restrictively or expansively), 51–52, 281–82. See also difficulty-adjusted probability scores
Bayesian belief-updating exercises, 252; ex ante assessments, 253–54; ex post assessments, 254–56; respondents, 252–53. See also reputational bets
Bayesians, 17, 18, 122, 123, 126n. 5, 129, 180
belief system defenses, 81, 129, 187, 224; challenging the conditions of hypothesis testing defense, 129–31; close-call counterfactual defense (“I was almost right”), 132–34, 140, 140n. 20, 141, 182; exogenous-shock defense, 131–32; “I made the right mistake” defense, 83, 135; just-off-on-timing defense, 134, 141; the low-productivity outcome just happened to happen defense, 135–36; playing-a-different-game defense, 186–87; politics is hopelessly cloudlike defense, 134–35; protecting, 156; quantitative analysis of, 136–37; really not incorrigibly closed-minded defense, 180–81; wrong questions defense, 184–85, 184–85n. 18
belief systems, minimalist versus maximalist models of constraint, 182–83, 183n. 16
belief-updating rule, violations of, 308–9, 311
Bell, D., 237n. 20
Berlin, Isaiah, 2, 2n. 3, 67, 72–73, 86–87, 88, 162, 241. See also hedgehog/fox dimension
Beyth-Marom, R., 123n. 2, 126n. 5
Bhagwati, J., 115n. 42
BJP Party, analogy to Nazi Party, 94, 94n. 22
Blake, William, 216
Blight, J., 5n. 7
Bloom, H., 23n. 44. See also meta-cognition (and the art of self-overhearing)
boomsters-doomsters, 71–72
Botha, P. W., 109
Braudel, Fernand, 144
Brazil, 114
Brehmer, B., 37n. 29
British Labor Party, 94
Bruner, Jerome, 226
Buffet, Warren, 33
Bulletin of the Atomic Scientists, xii
Bush, George W., 134; administration of, 6
butterfly effect arguments, 31–32
calibration, 47, 51–54, 72, 76, 78n. 9, 80n. 11, 84, 201, 275, 277; fox advantage of, 78–81, 79n. 10; versus discrimination, 279
Camerer, C., 65n. 50, 185n. 19, 232n. 15
Canada, 17, 115, 133, 134, 265–66. See also futures scenario experiments, Canadian
Carroll, J. S., 196n. 14
Carroll, Lewis, 216
Carter, Jimmy, 30
case-specific extrapolation algorithms, 51–53, 77, 282
case study methods (strengths and limitations), 4–7
Castro, Fidel, 5, 97
catch-up, 59, 83, 179–80
causality, 145–46
Cederman, L., 227n. 7
Chaiken, S., 118n. 45
“change scenarios,” 196
Chapman, G. B., 65n. 50
China, 29, 115–17, 186, 192, 203–4, 210; as “multiethnic empire,” 116
Chiu, C. Y., 160n. 16
Churchill, Winston, 26, 27
Civil War, the, 31
clairvoyance test, 13–14, 175, 243
clash of civilizations thesis, 105
Clausewitz, Carl von, 33n. 26
Clinton, Bill, 131, 132
close-call counterfactual exercises, xiv–xv, 258–59; beliefs about, 262–65; perceptions of close calls in Soviet history, 259–60; perceptions of contingency in South African history, 260–61; rewriting twentieth-century history, 261–62; unmaking the West, 264–65
cognitive conservatism, 7n. 11, 125–26, 126n. 5, 128, 128–29n. 9
cognitive process, 83–85
cognitive style, 75–76n. 7, 165, 182–83n. 16. See also hedgehog-fox dimension
Cohen, S., 148n. 7
coherence/process tests, 7, 17, 234
cold war, the, 155–56
communism: Chinese, 116, 157; communist regimes, collapse of, 96–97. See also Soviet Union, Communist Party of
complexity theorists, 30–32
conceptual integration index, 84
Congo, the, 106
conservatives, xiv n. 7, xv, xv n. 12, 12, 150–51, 152, 158–59, 257–58
constructivists/constructivism, 177, 179, 225–26
consumers, of expert pronouncements/information, 63, 231–32, 231n. 11, 235
correspondence indicators of good judgment, 7, 10, 12; calibration versus discrimination, 279; components of, 275, 277–78; overconfidence versus regression toward the mean, 279–80; —, extension to multiple outcomes, 281; —, and operationalizing “mindless” competition, 281–82; —, and operationalizing sophisticated competition, 282–83; probability score decomposition, 274–75; probability scoring, 273–74. See also probability scoring, adjustments to; value adjustments (of probability scores)
correspondence theory, 12, 15
counterfactuals, 18n. 32, 22, 140, 144–45, 156, 157, 161–63; plausible counterfactual reroutings of history, 145–48, 152–53; and reasoning as a two-stage affair, 147–48. See also close-call counter-factual exercises
covering laws, 88–91
Cowley, R., 31n. 17
Croatia, 91
Cromwell, Oliver, 143
Cuba, 97
Cuban missile crisis, xii, 5, 143, 206–9; research procedures for Cuban missile crisis experiment, 269–71
Czech Republic, 95
Dawes, Robyn, 12n. 18, 34, 37n. 30, 161n. 18
“debiasing” judgments, 185, 189–90; how checking one bias can amplify others, 213–15; of possible futures, 194–95; of possible pasts, 202–3, 212–13
Declaration of Independence, 89
de Klerk, Pieter Willem, 151, 152
Democrats, 6, 10n. 15, 13, 15
Deng Xiaoping, 93, 96, 108, 116, 176
deterrence theory, xiv Deutsch, M., xiii n. 5
difficulty-adjusted probability scores, 173–75, 284–88
dilettantes, 54, 56–57, 59
discrimination, 47, 51–54, 72, 84, 201, 277–78; versus calibration, 279. See also Normali
zed Discrimination Index (NDI)
dissonance, 39; belief-system defenses as modes of dissonance reduction, 129–37; “don’t bother me with dumb questions test,” 24; greater fox tolerance for dissonance as factor driving fox-hedgehog performance differentials, 84–85, 88, 141–42, 156, 160–61; neutralization of close-call counterfactuals as dissonance reduction, 147–63; neutralization of dissonant historical discoveries, 158–60
doomsters versus boomsters, 71–72
double standards, 181–82
dynamic–process test, 124–25. See also reputational bets
Eagly, A., 118n. 45
Edwards, W., 39n. 38, 126n. 5
Ehrlich, Paul, 15, 17
Einhorn, H., 21n. 40
Eliot, George, 25
Elstain, A., 65n. 50
Elster, J., 157n. 7
empirical accuracy, 190, 190n. 1, 218
empirical findings, manipulation of, 160n. 17
Ericsson, K. A., 65n. 50
Estonia, 95
Etheredge, L., xiv n. 7
“Eurocentric triumphalism,” 153
European Monetary Union (EMU), 89, 91, 132, 134
evaluative differentiation, 250–52; evaluative differentiation index, 83
Evans, P., 113n. 40
evidence, setting standards for, 156–61, 165, 190–91, 215
exclusivity and exhaustiveness test, 243–44
experts/expertise, 49, 54, 56–57, 59, 81, 122–23, 160n. 17, 193n. 9; accountability of, 185–86, 186n. 20; and framings of historical questions, 207–9; hedgehog, 82; and hindsight effects, 137–38; inaccurate predictions of, 161–62. See also forecasters/forecasting; futures scenario experiments
explanation, and prediction, 14
extrapolation algorithms: cautious or aggressive case-specific algorithms, 42, 53, 53n. 47; contemporaneous base rate algorithms, 51–52; formal statistical models, 42, 282–83; restrictive versus expansive recent base rates, 51–52;
extremists/extremism, 75, 80, 81, 82, 84. See also ideologue hypothesis; moderation-extremism
factor analysis, 69–71n. 1, 75n. 6
false alarms, 11–12; tendency for hedgehogs to issue false alarms of change, 83, 166–69. See also value adjustments (of probability scores)
falsification, 164, 180, 180n. 13
Farnham, B., 6n. 8
fatalism, 39
Fearon, J., 146n. 1
Feldstein, Martin, 15
Feyerabend, P., 3n. 4
Fischer, Stanley, 135
Fischhoff, B., 101n. 30, 123n. 2, 126n. 5, 191n. 2
Fiske, A., 112n. 38
Fiske, S., 37n. 30
Fitzgerald, F. Scott, 67
Fogel, Robert, 31
forecasters/forecasting, 14–15, 16, 17, 86n. 16, 166, 218n. 2, 223, 235n. 17; accuracy of, 33, 40–41, 81, 235, 249–52; and alternative perspectives, 122–23n. 1; and the forecasting horizon, 81, 82; inferiority of, 179; “inside” and “outside” approaches to, 194; methodological background, 44–49; political, 25; qualifications and motivations of, 185–86; regional, 20–21. See also belief system defenses; forecasting, types of questions; hedgehog/fox dimension; hypotheses; imagination effect; probability scoring; randomness; regional forecasting exercises; reputational bets; thought coding; value adjustments (of probability scores)
forecasting, types of questions, 246; continuity of domestic political leadership, 246–47, 249; criteria for, 243–44; domestic policy and economic performance, 247, 249–50; national security and defense policy, 247–48, 250; “posterior probability questions,” 255–56; special-purpose exercises, 248
Foucault, Michel, 225
Fox, C. R., 293n. 3
fox/foxes. See hedgehog/fox dimension
Friedman, T., xiii n. 6, 115n. 43
futures scenario experiments, 199–202, 199n. 15, 214, 253–54; Canadian, 195–98, 265–66; Japanese, 198–99, 266–67
Gaddis, J. L., 146n. 6
game theorists, 32–34; foxes’ greater sensitivity to stability/fragility of equilibria, 22, 107–12; foxes’ tendency to hedge bets on rationality, 112–17
Gandhi, Mahatma, 15, 26
Garb, H. N., 161n. 19
Garthoff, R., 93n. 21
Gartzke, E., 33n. 26
Gates, Bill, 17, 25
Geertz, Clifford, 230
Genco, T., 134n. 11
genocide, 12
George, A. L., 7n. 12
Georgia, 92
Gigerenzer, G., 21n. 39, 119n. 48, 183n. 17, 228n. 8
Gilbert, M., 26n. 4
Gilovich, T., 43n. 44
Gleick, J., 30n. 14
globalization, 115. See also boomsters-doomsters
Goldstone, J., 29n11
good judgment, 12n. 18, 23, 33, 144, 215, 217, 219, 220, 221, 227; and coherence and process tests, 7; and correspondence tests, 7; equated with good luck, 19–20; five challenges in assessing it, 11–13; obstacles to, 37; —, aversion to ambiguity and dissonance, 38–39; —, lightness of our understanding of randomness, 39–40; —, need for control, 39; —, preference for simplicity, 37–38; process and correspondence conceptions of, 141–43; and self-serving reasoning, 18. See also correspondence indicators of good judgment; good judgment, qualitative search for; good judgment, quantitative search for; leadership; logical-coherence, and process indicators of good judgment
good judgment, qualitative search for, 86–117 passim, 117–20
good judgment, quantitative search for, 68–86
passim, 117–20; cognitive style correlates, 72–73, 75–86; content correlates, 69, 71–72; demographic and life history correlates, 68
Gorbachev, Mikhail, xiii, xiv, xv, 12, 89, 96, 107–8, 131, 132
Gore, Al, 130, 131, 134
Gould, Stephen Jay, 26n. 2, 153
Great Depression, 186
Green, D., 10n. 15
Green, K., 86n. 16
Greenstein, F., 5n. 5
Grice, H. P., 122n. 1
Griffin, D., 123n. 2
gross domestic product (GDP), 247, 249–50
Grove, W. M., 54n. 48
Hammond, K., 7n. 10
Hastie, R., 138n. 16
Hawkins, S., 138n. 16, 191n. 4
Heath, C., 231n. 12
hedgehog/fox dimension, 2, 20–23, 72–73, 75–86, 75n. 6, 75–76n. 7, 79n. 10, 119n. 47, 127n. 7, 186–88, 241, 268; defense of hedgehogs, 164–66; “foxes are just chickens” hypothesis, 80, 85; and historical contingency, 205–6; integration of conflicting cognitions and foxes, 106–7; patterning of fox/hedgehog differences, 80–81; political passion, and foxes, 104–6; reasoning stages of hedgehog and fox, 88–92; rhetoric, and foxes, 100–101; “triumphalist” hedgehogs, 96; worries about judging the past, 101–4. See also calibration, fox advantage of; “debiasing” judgments; futures scenario experiments; good judgment; probability scoring; reputational bets; value adjustments (of probability scores)
Hempel, C., 90n. 18
“Hempelian” agenda, 90
Herrmann, R., xiv n. 7
heuristics, 119, 119–20n. 50, 236, 308
hindsight effects, 137–41, 162, 165; anticipatory, 102; hindsight bias, 140, 183–84, 183n. 17, 191, 203–5, 214; hindsight distortion, 138n. 18
Hiroshima and Nagasaki, 5
Hirt, E., 199n. 15
historians, 31, 101
history, 30, 38, 144, 145, 182–83; assessing causation in, 146; historical contingency, 205–6; and incomprehensibly intricate networks, 30–31; and narrative, 226–27; and technology, 29
Hitler, Adolf, 4, 10, 15, 26, 27, 99–100
hits, 11–12
Hogarth, R., 21n. 40, 185n. 19, 232n. 15
Huizinga, Johan, 144
Hume, David, 230
Hungary, 95
Hussein, Saddam, 1–2, 99, 106, 113–14, 114n. 41, 130, 133
hypotheses: debunking, 41–42, 49, 51–54; diminishing marginal returns from, 42–43, 54, 56–57, 59; “fifteen minutes of fame” (Andy Warhol hypothesis), 43, 59–60; indefinite
ly sustainable illusion, 43–44, 63–64; overconfidence (hot air hypothesis), 43, 60–62; seduced by power, 43, 62–63; systematic error, 61
hypothesis testing, 125–26n. 4, 162, 180; challenging the conditions of, 129–31
ideologue hypothesis, 75–76n. 7. See also moderation-extremism
“imaginability,” 197
imagination effect, 190–94, 195, 213–15; three conditions of for greatest effect, 196
indeterminacy, 32; and the “guess the number” game, 32–33. See also radical skepticism
index funds, 237
India, 93–94, 116
Indonesia, 90, 106
institutionalists, 71, 90
integrative complexity index, 84, 252
International Monetary Fund (IMF), 105, 114, 135
Iraq, 106, 114n. 41; U.S. invasion of, 1–2
Islam, 29, 210
Iyengar, S., 76n. 7
Jacobson, H., 113n. 40
Japan, 5, 110–11, 266–67. See also futures scenario experiments, Japanese
Jensen’s inequality, 179
Jentleson, B., 4n. 5
Jervis, Robert, 38n. 31, 40n. 41, 100n. 29, 134n. 11, 173n. 9
John, O., 75n. 6
Johnson, E., 65n. 50
Johnson, Samuel, 4, 224
Jost, J., 75n. 7
judgment. See “debiasing” judgments; good judgment
Kahneman, Daniel, 40n. 41, 189, 194, 293n. 3
Kashmir, 99, 101
Kazakhstan, 94–95, 133
Kennedy, John F., 5
Kenny, D., 137n. 14
Kent, Sherman, 17n. 29, 27n. 29, 143n. 22, 238n. 22
Keren, G., 66n. 50
Keynes, John Meynard, 121
Khong, Y. F., 4n. 5
Kim Jong-Il, 96, 99
King Fahd, 98
Kirkpatrick, Jeanne, xv
Kissinger, Henry, 134
k method (of value-adjusting probability scores), the, 57, 169, 288, 289–92
knowledge, “ideographic” and “nomothetic,” 8n. 13
Koehler, J., 40n. 40, 303n. 5
Kruglanski, A., 11n. 16, 75nn. 6 and 7, 126n. 6, 160n. 16
Krugman, P., 115n. 42
Kunda, Z., 2n. 2, 128n. 9
Lakatos, I., 137n. 15, 180n. 13
Langer, E., 39n. 37
Larsen, D., 5n. 5
Laudan, P., 18n. 31
leaders: rationality of, 112–13; “when do leaders matter?” 107–12
leadership, 18n. 32, 107, 107n. 34
Lebow, Ned, 206
left versus right, 71, 75–76n. 7
Legvold, R., 109n. 35
Lepper, M., 128n. 8
level playing fields, 11
Expert Political Judgment Page 47