What Intelligence Tests Miss

by Keith E Stanovich

  5 Why Intelligent People Doing Foolish Things Is No Surprise

  1. The figures used here are from Zweig (2002).

  2. It appears that loss aversion (see Kahneman and Tversky, 1979) is an affective forecasting error—when the events actually occur, the aversive valence of the loss is in fact not twice that of the gain (Kermer, Driver-Linn, Wilson, and Gilbert, 2006). On myopic loss aversion see Thaler, Tversky, Kahneman, and Schwartz (1997). On the tendency to explain chance events, particularly those that occur in markets, see Malkiel (2004), Nickerson (2004), and Taleb (2001, 2007).

  3. Several classic papers in psychology established the idea of humans as cognitive misers (Dawes, 1976; Simon, 1955, 1956; Taylor, 1981; Tversky and Kahneman, 1974).

  4. Of course, evolution guarantees rationality in the dictionary sense of “the quality or state of being able to reason” because evolution built the human brain. What I mean here is that evolution does not guarantee rationality in the sense the term is used throughout cognitive science—as maximizing subjective expected utility (Gauthier, 1975). There is a literature on the nature of human long-term interests and their possible divergence from the short-term strategies of evolutionary adaptation (Ainslie, 2001; de Sousa, 2007; Haslam and Baron, 1994; Loewenstein, 1996; Nozick, 1993; Oatley, 1992; Parfit, 1984; Pinker, 1997; Sabini and Silver, 1998; Stanovich, 2004). On natural selection as a “better than” mechanism, see Cosmides and Tooby (1996, p. 11). Ridley (2000) spins this point another way, calling evolution “short-termish” because it is concerned with immediate advantage rather than long-term strategy. Human rationality, in contrast, must incorporate the long-term interests of the individual.

  5. On affective forecasting, see Gilbert (2006), Kahneman, Diener, and Schwarz (1999), and Wilson and Gilbert (2005).

  6. On the culturally derived nature of rational standards, see Jepson, Krantz, and Nisbett (1983), Krantz (1981), and Thagard and Nisbett (1983). On changes in the environment and their implications for fitness and human goals, see Richerson and Boyd (2005) and Stanovich (2004).

  7. Being a cognitive miser is the universal default in naturalistic situations. When cued that more intense cognitive effort is necessary, those of higher intelligence will have an advantage due to their greater computational ability.

  8. Two critical caveats are in order here. Most (but not all) of the studies that I cite in this book employed university students as subjects. The higher and lower intelligence groups that I am discussing are, in most instances, partitionings of the upper half and lower half of the subject sample. Thus, the lower-IQ individuals are not low-IQ in an absolute sense. They simply are of lower intelligence relative to their counterparts in the particular study. The second point is related to the first. The magnitude of the correlations involving intelligence obtained in these investigations is undoubtedly attenuated because of restriction of range. Again, this is because most investigations employed university students as subjects. Nevertheless, this caveat about attenuation itself needs contextualization. Certainly, it is true that individuals with average and above average cognitive ability are over-represented in samples composed entirely of university students. Nevertheless, the actual range in cognitive ability found among college students in the United States is quite large. In the past 30 years, the percentage of 25- to-29-years-olds in the United States who have attended college has increased by 50 percent. By 2002, 58 percent of these young adults had completed at least one or more years of college, and 29 percent had received at least a bachelor’s degree (Trends, 2003). Finally, the fact that the range of the samples studied is somewhat restricted makes many of the findings (of near zero correlations between intelligence and rational thought) no less startling. It is quite unexpected that, across even the upper two thirds of cognitive ability, there would be little relation between rational thought and intelligence.

  9. See Postman (1988, pp. 86–87).

  6 The Cognitive Miser

  1. The Anne problem, and others like it, are discussed in Levesque (1986, 1989), and our work on these types of problem is discussed in Toplak and Stanovich (2002). On disjunctive reasoning, see Johnson-Laird (2006), Shafir (1994), and Toplak and Stanovich (2002). The tendency toward default processing that is computationally simple is not restricted to problems that are themselves simple. It is displayed in more complex problems as well (see Evans, 2007; Kahneman, 2003a; Stanovich, 1999, 2004; Taleb, 2007).

  2. The bat and ball problem is described in Kahneman and Frederick (2002) and the studies of MIT, Princeton, and Harvard students in Frederick (2005).

  3. See Kahneman (2003a) on accessibility and how it is substituted for more complexjudgments of probability. On the California earthquake example, see Kahneman and Frederick (2002), and Tversky and Kahneman (1983).

  4. The literature on affective valuation is extensive (e.g., Forgas, 1995; Frederick, 2002; Loewenstein, Weber, Hsee, and Welch, 2001; Oatley, 1992, 2004; Rottenstreich and Hsee, 2001; Schwarz and Clore, 2003; Slovic, Finucane, Peters, and MacGregor, 2002; Slovic and Peters, 2006). On the shock study, see Rottenstreich and Hsee (2001). On public valuation studies, see Kahneman and Frederick (2002). The panda study is from Hsee and Rottenstreich (2004).

  5. There are numerous ways to calculate travel risk, but driving consistently looks extremely dangerous across various metrics (Galovski, Malta, and Blanchard, 2006; National Safety Council, 1990, 2001; Sivak, 2002; Sivak and Flannagan, 2003; Sunstein, 2002). The post–September 11, 2001, travel statistics are from Gigerenzer (2004) and Sivak and Flanagan (2003). On diabetes versus staph infections, see Fountain (2006).

  6. Yamagishi (1997). On the effect of vividness, see Slovic (2007).

  7. On the money illusion, see Kahneman, Knetsch, and Thaler (1986) and Shafir, Diamond, and Tversky (1997). Raghubir and Srivastava (2002) reported the foreign currency study. Wertenbroch, Soman, and Chattopadhyay (2007) have shown that the face value effect can be dependent on budgetary limitations that people use as reference points. Their findings complexify our understanding of the money illusion but do not change my point here—that the money illusion is an example of cognitive miser tendencies in human information processing.

  8. On the usefulness of heuristic processing, see Gigerenzer (2007); Gladwell (2005); Klein (1998); McKenzie (1994); Pinker (1997); Todd and Gigerenzer (2007).

  9. Classic work in the psychology of anchoring is described in Tversky and Kahneman (1974). As always in cognitive psychology, after the initial discovery of an important phenomenon, our understanding of the phenomenon quickly “complexifies.” For example, sometimes anchoring appears to derive from insufficient adjustment from an anchor, and other times it is due to the increased accessibility of anchor-consistent information (the former when the anchor is self-generated and the latter in the standard paradigm; see Epley and Gilovich, 2006). A more fine-grained view of how anchoring and adjustment works is provided in many other publications (see Brewer and Chapman, 2002; Epley and Gilovich, 2004, 2006; Jacowitz and Kahneman, 1995; Jasper and Chirstman, 2005; LeBoeuf and Shafir, 2006; Mussweiler and Englich, 2005; Mussweiler, Englich, and Strack, 2004; Wilson, Houston, Etling, and Brekke, 1996). The nuances surrounding our current understanding of anchoring effects have no bearing on the very basic points I make about anchoring in this book. The study of the real estate agents is reported in Northcraft and Neale (1987). For the study of actual judges being affected by anchoring, see Englich, Mussweiler, and Strack (2006).

  10. Several studies have shown the less-is-more context effect (Bartels, 2006; Slovic et al., 2002; Slovic and Peters, 2006). On evaluability, see Hsee (1996); Hsee, Loewenstein, Blount, and Bazerman (1999); and Hsee and Zhang (2004).

  11. Todd and Gigerenzer (2007) use the term default heuristic. Several important papers discuss work on the status quo bias (Frederick, 2002; Hartman, Doane, and Woo, 1991; Kahneman, Knetsch, and Thaler, 1991; Samuelson and Zeckhauser, 1988; Thaler, 1980). It should be emphasized that it is an unthinking overuse of the default heuristic that is
irrational. Many theorists have pointed out that in some situations it is rational to view defaults as the recommendations of a policy maker (Johnson and Goldstein, 2006; McKenzie, Liersch, and Finkelstein, 2006; Sunstein and Thaler, 2003).

  12. Gigerenzer (2002, 2007; Brandstatter, Gigerenzer, and Hertwig, 2006; Gigerenzer and Goldstein, 1996; Todd and Gigerenzer, 2000, 2007) is an influential champion of this view (in rebuttal, see Evans, 2007; Kahneman and Tversky, 1996; Over, 2000; Stanovich, 2004). However, whether the heuristics studied by the Gigerenzer group fit the category of Type 1 processes is very doubtful (see Evans, 2007, and Kahneman and Frederick, 2002, for a discussion). Sterelny (2003) has written perceptively about the evolutionary significance of hostile environments. Writer Louis Menand (2004), in discussing work on the heuristics people use for voting decisions, contextualizes the use of heuristics in the manner that I do here: “Any time information is lacking or uncertain, a shortcut is generally better than nothing. But the shortcut itself is not a faster way of doing the math; it’s a way of skipping the math altogether. My hunch that the coolest-looking stereo component is the best value simply does not reflect an intuitive grasp of electronics. My interest in a stereo is best served if I choose the finest sound for the money, as my interest in an election is best served if I choose the candidate whose policies are most likely to benefit me or the people I care about” (p. 95).

  13. The chapter subtitled “How Ignorance Makes Us Smart” is in Gigerenzer and Todd (1999). The Wimbledon study is described in Todd and Gigerenzer (2007). See also Goldstein and Gigerenzer (1999, 2002). Bazerman (2001) discusses proper personal finance strategies, and the British bank example is in MacErlean (2002, p. 2).

  14. Here is an example of how advocates of rational thinking often get caricatured. By the phrase “think through the alternatives” I obviously do not mean an exhaustive comparison and contrast of each of the thousands of mutual funds on offer. Instead, I mean thinking through each of the major classes of decision in this domain: load versus no-load funds; index funds versus managed funds; bond, stock, and cash allocation; the amount of foreign exposure; the amount of real estate and commodities exposure; etc. It is a common strategy to denigrate explicit rational thought by showing that a particular decision situation involves a combinatorial explosion beyond even what the largest computer could handle (pairwise comparison of 6000 mutual funds on X dimensions from Y different perspectives on Z different personal financial goals). But the advocates of rational strategies have no such ludicrous mechanical procedure in mind. Just because the comparison of thousands of funds is unfeasible does not mean that we should rely on a quick and dirty heuristic response in this important domain of personal finance. Between the exhaustive comparison and the quick and dirty heuristic is a middle ground where, in this domain for example, we would engage in extended explicit thought about a few key variables: tolerance for risk, age, current assets and debts, income needed in retirement, and a few other key factors.

  15. Sinaceur, heath, and Cole (2005).

  7 Framing and the Cognitive Miser

  1. See McCaffery and Baron (2004, 2006a, 2006b, especially 2006b) for discussions of their studies. My example is a simplified variant of the type of problem that appeared in their experiments. The child deduction example was originally discussed in Schelling (1984, pp. 18–20).

  2. The rule of descriptive invariance is, according to Tversky and Kahneman (1986), that “variations of form that do not affect the actual outcomes should not affect the choice” (p. 253). For further discussions of descriptive invariance see Kahneman and Tversky (1984, 2000) and Tversky and Kahneman (1981). Beyond descriptive invariance, utility maximization requires adherence to a set of further axioms of choice (see Allingham, 2002; Dawes, 1998; Edwards, 1954; Jeffrey, 1983; Luce and Raiffa, 1957; Savage, 1954; von Neumann and Morgenstern, 1944; see Wu et al., 2004, for a review).

  3. There have been several important papers on the equality heuristic (e.g., Frederick, 2002; Harris and Joyce, 1980; Messick, 1999; Messick and Schell, 1992).

  4. The transplant study is discussed in Ubel (2000).

  5. On the cognitive miser tendency to take a problem representation as “given,” and on other aspects of framing effects discussed in this chapter, see the voluminous literature on framing effects in decision science (Kahneman and Tversky, 1984, 2000; Kuhberger, 1998; LeBoeuf and Shafir, 2003; Levin et al., 2002; Maule and Villejoubert, 2007; McElroy and Seta, 2003; Simon, Fagley, and Halleran, 2004; Slovic, 1995; Tversky and Kahneman, 1981, 1986). This literature describes many paradigms not involving gambling at all and several involving real-world content (Epley, Mak, and Chen Idson, 2006; Friedrich, Lucas, and Hodell, 2005; McNeil, Pauker, Sox, and Tversky, 1982; Schneider, Burke, Solomonson, and Laurion, 2005). Decisions 1 and 2 are from Tversky and Kahneman (1986).

  6. On coding options from a zero reference point, see also Markowitz (1952), and on the valuation of good and bad outcomes see Baumeister, Bratslavsky, Finkenauer, and Vohs (2001). Other key features of prospect theory are covered in Kahneman and Tversky (1979) and Tversky and Kahneman (1986, 1992).

  7. Epley, Mak, and Chen Idson (2006). See also Epley (2008).

  8. Thaler (1980). The operation of the default heuristic in insurance decisions is described in Johnson, Hershey, Meszaros, and Kunreuther (2000).

  9. Friedrich, Lucas, and Hodell (2005).

  10. Individual differences work on framing using both types of designs is not extensive, but the literature is growing (see Bruine de Bruin et al., 2007; Frederick, 2005; Leboeuf and Shafir, 2003; Parker and Fischhoff, 2005; Stanovich and West, 1998b, 1999, 2008b).

  11. I will repeat here the caveat that the higher- and lower-intelligence groups that I am often discussing are, in most instances, partitionings of the upper half and lower half of a university sample. Thus, the lower-IQ individuals are not low-IQ in an absolute sense. Also, the magnitude of the correlations involving intelligence obtained in these investigations is undoubtedly attenuated because of restriction of range. Nonetheless, it is still quite striking that, across even the upper two-thirds of cognitive ability, there would be so little relation between crucial aspects of rational thought and intelligence.

  12. See Bruine de Bruin et al. (2007); Parker and Fischhoff (2005); Stanovich and West (1999; 2008b); Toplak and Stanovich (2002).

  8 Myside Processing

  1. For the relevant crash statistics at the time the study was conducted, see NHTSA (2000), Vehicle design versus aggressivity, National Highway Traffic Safety Administration, U.S. Department of Transportation (DOT HS 809 194), Retrieved February 23, 2002, from NHTSAwebsite http://www-nrd.nhtsa.dot.gov/pdf/nrd-11/DOT_HS_809194.pdf.

  2. Our study is reported in Stanovich and West (2008a). The Westen study is reported in Westen, Blagov, Kilts, and Hamann (2006). Related paradigms have been studied by our lab (Stanovich and West, 2007) and are discussed in a variety of sources (Kunda, 1990, 1999; Mele, 2003; Molden and Higgins, 2005; Perkins, Farady, and Bushey, 1991; Thagard, 2006). On the argument generation paradigm described in this chapter, see Baron (1995), Macpherson and Stanovich (2007), Perkins (1985), Toplak and Stanovich (2003).

  3. The experiment evaluation paradigm has generated a small literature (Klaczynski, 1997; Klaczynski and Gordon, 1996; Klaczynski, Gordon, and Fauth, 1997; Klaczynski and Lavallee, 2005; Klaczynski and Robinson, 2000; Macpherson and Stanovich, 2007). Educational psychologist Deanna Kuhn has developed a structured interview to study myside bias in informal reasoning (Kuhn, 1991, 1992, 1993). Our study using the Kuhnian interview is reported in Sá et al. (2005).

  4. On otherside processing being demanding, see Gilbert, Pelham, and Krull (1988). Taber and Lodge (2006) report one of the more comprehensive studies of various aspects of myside processing.

  5. The knowledge calibration paradigm specifically, and belief calibration in general, have undergone three decades of research (e.g., Fischhoff, Slovic, and Lichtenstein, 1977; Griffin and Tversky, 1992; Koriat, Lichtenstein, and Fis
chhoff, 1980; Lichtenstein and Fischhoff, 1977; Schaefer, Williams, Goodie, and Campbell, 2004; Sieck and Arkes, 2005; Tetlock, 2005; Yates, Lee, and Bush, 1997). This literature and itsmethodological complexities have been reviewed in several sources (Baron, 2000; Fischhoff, 1988; Griffin and Varey, 1996; Lichtenstein, Fischhoff, and Phillips, 1982).

  6. These five questions were taken from Plous (1993) and Russo and Schoemaker (1989).

  7. Overconfidence effects have been found in perceptual and motor domains (Baranski and Petrusic, 1994, 1995; West and Stanovich, 1997; Wright and Ayton, 1994), sports outcomes (Ronis and Yates, 1987), reading comprehension monitoring (Pressley and Ghatala, 1990), judging the sex of handwriting samples (Schneider, 1995), prediction of one’s own behavior or life outcomes (Hoch, 1985; Vallone, Griffin, Lin, and Ross, 1990), and economic forecasts and political predictions (Åstebro, Jeffrey, and Adomdza, 2007; Braun and Yaniv, 1992; Tetlock, 2005). On the planning fallacy, see Buehler, Griffin, and Ross (2002). The Kahneman anecdote is from “a short Course in Thinking About Thinking” by Daniel Kahneman, retrieved on 9/27/07 from http://www.edge.org/3rd_culture/kahneman07/kahneman07_index.html.

  8. Cell phone use—even the use of hands-free phones—impairs driving ability to an extent that substantially increases the probability of an accident (McEvoy et al., 2005; Strayer and Drews, 2007; Strayer and Johnston, 2001). The Canada Safety Council study is discussed in Perreaux (2001). On most drivers thinking they are above average, see Svenson (1981). Groopman (2007) discusses overconfidence among physicians.

  9. The study of the 800,000 students is described by Friedrich (1996). See Kruger and Dunning (1999) for the study of the test takers. Biased self-assessment research has many methodological and statistical complexities that are well discussed by Moore (2007) and by Larrick, Burson, and Soll (2007). Many of the earlier interpretations of this research are undergoing renewed debate. Nonetheless, for a sampling of the research that I am drawing from, see: Dunning, Heath, and Suls (2004); Dunning, Johnson, Ehrlinger and Kruger (2003); Friedrich (1996); Kruger and Dunning (1999); Larrick et al. (2007); Moore and Small (2007); and Myers (1990). Larrick et al. (2007) and Moore and Small (2007) discuss the complex issue of how overestimates of one’s own performance are related to overestimates of one’s own performance relative to others. Regardless of the outcome of these theoretical disputes, both phenomena seem to result from myside processing that always makes one’s own beliefs the focal model for subsequent processing.