by Ian Ayres
People guess poorly: J. Edward Russo and Paul J. H. Schoemaker, Decision Traps: Ten Barriers to Brilliant Decision-Making and How to Overcome Them (1990). See also Scott Plous, The Psychology of Judgment and Decision Making (2003); John Ruscio, “The Perils of Post-Hockery: Interpretations of Alleged Phenomena After the Fact,” Skeptical Inquirer, Nov.–Dec. 1998.
Costs of the Iraq War: Interview with Vice President Cheney, CNN broadcast, Jun. 20, 2005;. interview with Glenn Hubbard, CNBC broadcast, October 4, 2002; Reuters, “U.S. Officials Play Down Iraq Reconstruction Needs,” Entous, Apr. 11, 2003; Hearing on a Supplemental War Regulation Before the H. Comm. on Appropriations, 108th Cong. (Mar. 27, 2003) (statement of Deputy Defense Sec’y Paul Wolfowitz); Rep. Jan Schakowsky, “Past Comments About How Much Iraq Would Cost,” www.house.gov/schakowsky/iraqquotes_web.htm.
Human judges: Richard Nisbett and Lee Ross, Human Inference: Strategies and Shortcomings of Social Judgment (1980).
Super Crunching without emotions: Douglas Heingartner, “Maybe We Should Leave That Up to the Computer,” N.Y. Times, Jul. 18, 2006.
Can Super Crunchers and experts coexist?: S. Schwartz et al., “Clinical Expert Systems Versus Linear-Models: Do We Really Have to Choose,” 34 Behavioral Sci. 305 (1989).
Humans serving machines: Douglas Heingartner, “Maybe We Should Leave That Up to the Computer,” N.Y. Times, Jul. 18, 2006.
Parole predictions: For an early study assessing the likely outcomes for parolees, see Earnest W. Burgess, “Factors Determining Success or Failure on Parole,” in The Workings of the Indeterminate Sentence Law and the Parole System in Illinois (A. A. Bruce, ed., 1928), pp. 205–249. For biographical information and account of Burgess’s place in sociology, use of new methods of measurement, see Howard W. Odum, American Sociology: The Story of Sociology in the United States Through 1950 (1951), pp. 168–171, http://www2.asanet.org/governance/burgess.htm.
Clouston and the SVPA: Virginia General Statutes, § 37.1–70.4 (C); Frank Green, “Where Is This Man?: Should This Child Molester and Cop Killer Have Been Released?” Richmond Times-Dispatch, Apr. 16, 2006.
Supremes uphold SVPA: Kansas v. Hendricks, 521 U.S. 346 (1997).
Monahan on the first Super Crunching statutory trigger: Bernard E. Harcourt, Against Prediction: Profiling, Policing and Punishment in an Actuarial Age (2007); John Monahan, Forecasting Harm: The Law and Science of Risk Assessment among Prisoners, Predators, and Patients, ExpressO Preprint Series (2004), http://law.bepress.com/expresso/eps/410.
Conditioning commitment on legal, but statistically predictive, behavior: Eugene Volokh on his blog, The Volokh Conspiracy, has questioned whether the sex discrimination would withstand constitutional scrutiny. Eugene Volokh, “Sex Crime and Sex,” The Volokh Conspiracy, Jul. 14, 2005, http://volokh.com/ posts/1121383012.shtml. I do have concerns about whether the RRASOR system is in fact statistically valid. The original dataset upon which it was estimated was “artificially reduced” to 1,000 observations to intentionally reduce the statistical significance of the explanatory variables. R. Karl Hanson, The Development of a Brief Actuarial Scale for Sexual Offense Recidivism (1997). Instead of constructing the crude point system, a more tailored approach would estimate recidivism probabilities based directly on the coefficients of the regression equation. The constructors of RRASOR seemed to be operating in a pre-computer environment where assessments had to be easily calculated by hand.
Introduction of the broken-leg case: Paul E. Meehl, Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence (1954) (reissued University of Minnesota 1996).
Tom Wolfe: Tom Wolfe, The Right Stuff (1979).
Mistaken discretionary releases: Frank Green, “Where Is This Man?: Should This Child Molester and Cop Killer Have Been Released?” Richmond Times-Dispatch, Apr. 16, 2006.
Assessing human overrides: James M. Byrne and April Pattavina, “Assessing the Role of Clinical and Actuarial Risk Assessment in an Evidence-Based Community Corrections System: Issues to Consider,” Fed. Probation (Sep. 2006). See also Laurence L. Motiuk et al., “Federal Offender Population Movement: A Study of Minimum-security Placements,” Correctional Service of Canada, (Mar. 2001); Patricia M. Harris, “What Community Supervision Officers Need to Know About Actuarial Risk Assessment and Clinical Judgment,” Fed. Probation (Sep. 2006).
The work that’s left for humans: Drew Western and Joel Weinberger, “In Praise of Clinical Judgment: Meehl’s Forgotten Legacy,” 61 J. Clinical Psychol. 1257, 1259 (2005); Paul E. Meehl, “What Can the Clinician Do Well?” in Problems in Human Assessment 594 (D. N. Jackson and S. Messick, eds., 1967); PaulE. Meehl, “Causes and Effects of My Disturbing Little Book,” 50 J. Personality Assessment 370 (1986).
Fink’s circumcision hypothesis: A. J. Fink, Letter, “A Possible Explanation for Heterosexual Male Infection with AIDS,” 315 N. Engl. J. Med. 1167 (1986).
HIV-transmission hypotheses: Cameron D. William et al., “Female to Male Transmission of Human Immunodeficiency Virus Type 1: Risk Factors for Seroconversion in Men,” 2 Lancet 403 (1989). See also J. Simonsen et al., “Human Immunodeficiency Virus Infection in Men with Sexually Transmitted Diseases,” 319 N. Engl. J. Med. 274 (1988); M. Fischl et al., “Seroprevalence and Risks of HIV Infection in Spouses of Persons Infected with HIV,” Book 1, 4th Int’l Conf. on AIDS 274, Stockholm, Sweden (Jun. 12–16, 1988).
Continuing empiricism on the circumcision-AIDS connection: D. T. Halperin et al., “Male Circumcision and HIV Infection: 10 Years and Counting,” 354 Lancet 1813 (1999); Donald G. McNeil Jr., “Circumcision’s Anti-AIDS Effect Found Greater Than First Thought,” N.Y. Times, Feb. 23, 2007.
Weight loss investments: I personally have signed a contract putting thousands of my dollars at risk if I don’t take off twenty pounds this year and keep it off. Dean Karlan and I are trying to put together a randomized study of weight-loss bonds. Send me an email at [email protected] if you’re interested in participating.
Hammond on clinical resistance: Kenneth R. Hammond, Human Judgment and Social Policy 137–38 (1996).
CHAPTER 6
Google Books creates a virtual library: Jeffrey Toobin, “Google’s Moon Shot,” New Yorker, Feb. 5, 2007.
Discriminatory negotiation practices: These results came from my initial study: Ian Ayres, “Fair Driving: Gender and Race Discrimination in Retail Car Negotiations,” 104 Harv. L. Rev. 817 (1991). An analysis of five more recent studies can be found in my book Pervasive Prejudice?: Non-Traditional Evidence of Race and Gender Discrimination (2001).
Discriminatory auto-lending mark-ups: Ian Ayres, “Market Power and Inequality: A Competitive Conduct Standard for Assessing When Disparate Impacts Are Justified,” Cal. L. Rev. (2007).
ChoicePoint sells data: Gary Rivlin, “Keeping Your Enemies Close,” N.Y. Times, Nov. 12, 2006; ChoicePoint 2005 Annual Report, http://library. corporate-ir.net/library/95/952/95293/items/189639/2005annual.pdf.
Biggest company you never heard of: “Persuaders,” PBS Frontline, http://www.pbs.org/wgbh/pages/frontline/shows/persuaders/etc/script.htm; see also Richard Behar, “Never Heard of Acxiom? Chances Are It’s Heard of You,” Fortune, Feb. 23, 2004.
134: 850 terabytes of storage: Rick Whiting, “Tower of Power,” InformationWeek, Feb. 11, 2002.
Data silos: Kim Nash, “Merging Data Silos,” Computerworld, April 15, 2002; Gary Rivlin, “Keeping Your Enemies Close,” N.Y. Times, Nov. 12, 2006; Eric K. Neumann, “Freeing Data, Keeping Structure,” Bio-IT World, Jun. 14, 2006.
Data “mashups”: Rachel Rosmarin, “Maps. Mash-ups. Money.” Forbes.com, Jun. 16, 2006, http://www.forbes.com/technology/2006/06/14/google-yahoomicrosoft_cx_rr_0615maps.htm.
Car theft database: This data is available from the Federal Bureau of Investigation’s National Crime Information Center (NCIC), http://www. fas.org/irp/agency/doj/fbi/is/ncic.htm.
Erroneous felon disenfranchisement: The United States Civil Rights Commission, The 2000 Presidential Elections (www.usccr.gov/
pubs/vote2000/ report/ch5.htm).
Kryder’s Law: “Kryder’s Law” is the title of an article by Chip Walter in the Aug. 2005 issue of Scientific American.
Computer storage capacity costs decrease over time: “Historical Notes about the Cost of Hard Drive Storage Space,” www.littletechshoppe.com/ ns1625/winchest.htm; Jim Handy, “Flash Memory vs. Hard Disk Drives—Which Will Win?” Jun. 6, 2005, http://www.storagesearch.com/semicoart1.htm.
Twelve terabytes for Yahoo: Kevin J. Delaney, “Lab Test: Hoping to Overtake Its Rivals, Yahoo Stocks Up on Academics,” Wall St. J., Aug. 25, 2006, p. Al.
Background on neural networks: P. L. Brockett et al., “A Neural Network Method for Obtaining an Early Warning of Insurer Insolvency,” 61 J. Risk and Insurance 402 (1994); Jack V. Tu, “Advantages and Disadvantages of Using Artificial Neural Networks versus Logistic Regression for Predicting Medical Outcomes,” 50 J. Clin. Epidemiol. 1309 (1996); W. G. Baxt, “Analysis of the Clinical Variables Driving Decision in an Artificial Neural Network Trained to Identify the Presence of Myocardial Infarction,” 21 Ann. Emerg. Med. 1439(1992); K. A. Spackman, “Combining Logistic Regression and Neural Networks to Create Predictive Models,” in Proceedings of the Sixteenth Annual Symposium on Computer Applications in Medical Care (1992), pp. 456–59; J. L. Griffith et al., “Statistical Regression Techniques for the Construction, Interpretation and Testing of Computer Neural Networks,” 12 Med. Decis. Making 343 (1992).
Neural networks and greyhound racing: Hsinchun Chen et al., “Expert Prediction, Symbolic Learning, and Neural Networks: An Experiment on Greyhound Racing,” 9 IEEE Expert 21 (Dec. 1994).
Epagogix revealed: Malcolm Gladwell, “The Formula,” New Yorker, Oct. 16, 2006.
Lulu scores book titles: Misty Harris, “Anyone Who Says You Can’t Judge a Book by Its Cover Isn’t Trying Hard Enough,” Windsor Star, Dec. 17, 2005, http://www.canada.com/windsorstar/features/onlineextras/story.htm?id= 35711d6b-f13e-47d9-aa23-7eaa12bc8846.
What makes a law review article likely to be cited?: Ian Ayres and Fredrick E. Vars, “Determinants of Citations to Articles in Elite Law Reviews,” 29 J. Legal Stud. 427, 433–34 (2000).
Just stick the right formula in: Indigo Girls, “Least Complicated,” on Swamp Ophelia (1994).
Robert Frost on walls: Robert Frost famously wrote, “Something there is that doesn’t love a wall/That wants it down.” Robert Frost, “Mending Wall”(1915).
Library of Alexandria several times over on the Internet: Kevin Kelly, “Scan this Book!” N.Y. Times Magazine, May 14, 2006, p. 43.
Ubiquitous surveillance: A. Michael Froomkin, “The Death of Privacy,” 52 Stan. L. Rev. 1461 (2000).
Nanotechnological advances: George Elvin, “The Coming Age of Nanosensors,” NanotechBuzz, http://www.nanotechbuzz.com/50226711/the_ coming_age_of_nanosensors.php.
Smart dust: Gregor Wolbring, “The Choice Is Yours: Smart Dust,” Innovation Watch, Dec. 15, 2006, http://www.innovationwatch.com/choiceisyours/choiceis yours.2006.12.15.htm.
CHAPTER 7
Ms. Daniel’s lesson: Siegfried Engelmann and Elaine C. Bruner, “The Pet Goat,” in Reading Mastery II: Storybook 1 (1997). See also Daniel Radosh, “The Pet Goat Approach,” New Yorker, Jul. 26, 2004.
The goat story on the silver screen: Fahrenheit 9/11 (Sony Pictures 2004); Daniel Radosh, “The Pet Goat Approach,” New Yorker, Jul. 26, 2004.
The controversy over Direct Instruction: W. C. Becker, “Direct Instruction: A Twenty Year Review,” in Designs for Excellence in Education: The Legacy of B. F. Skinner (R. P. West and L. A. Hamerlynck, eds., 1992), pp. 71–112;G. L. Adams and S. Engelmann, Research on Direct Instruction: 25 Years beyond DIS TAR (1996); Am. Fed. Teachers, Direct Instruction (1998); American Institutes for Research, Comprehensive School Reform Quality Center, CSRQ Center Report on Elementary School Comprehensive School Reform Models (2006), http:// www.csrq.org/documents/CSRQCenterCombinedReport_Web11-03-06.pdf; Jean Piaget, Adaptation and Intelligence: Organic Selection and Phenocopy (1980); Linda B. Stebbins et al., Education as Experimentation: A Planned Variation Model Volume IV-A An Evaluation of Project Follow Through (1977); Sanjay Baht, “A New Way of Judging How Well Schools Are Doing,” Seattle Times, Aug. 2, 2005; Ted Hershberg et al., “The Revelations of Value-Added,” School Administrator, Dec. 2004; Siegfried Engelmann, War Against the Schools’ Academic Child Abuse (1992); David Glenn, “No Classroom Left Unstudied,” The Chronicle of Higher Education, May 28, 2004; Richard Nadler, “Failing Grade,” Nat’l Rev., Jun. 1, 1998; Am. Fed. Teachers, Building on the Best, Learning from What Works: Six Promising Schoolwide Reform Programs (1998); B. Gunn et al., “The Efficacy of Supplemental Instruction in Decoding Skills for Hispanic and Non-Hispanic Students in Early Elementary School,” 34 J. Special Ed. 90 (2000); B. Gunn et al., “Supplemental Instruction in Decoding Skills for Hispanic and Non-Hispanic Students in Early Elementary School: A Follow-Up,” 36 J. Special Ed. 69 (2002).
Some samples of Engelmann’s work: G. L. Adams and S. Engelmann, Research on Direct Instruction: 25 Years beyond DISTAR (1996); Siegfried Engelmann, War Against the Schools’ Academic Child Abuse (1992). See also Daniel Radosh, “The Pet Goat Approach,” New Yorker, Jul. 26, 2004.
Piaget’s child-centered approach: Jean Piaget, Adaptation and Intelligence: Organic Selection and Phenocopy (1980).
Project Follow Through: Bonnie Grossen (ed.), “Overview: The Story Behind Project Follow Through,” Effective School Practices, 15:1, Winter 1995–96, http://darkwing.uoregon.edu/~adiep/ft/grossen.htm.
DI wins hands down: Richard Nadler, “Failing Grade,” Nat’l Rev., Jun. 1, 1998, http://www.nationalreview.com/01jun98/nadler060198.htm.
Recent studies support DI: Am. Fed. Teachers, Building on the Best, Learning from What Works: Six Promising Schoolwide Reform Programs (1998); B. Gunn et al., “The Efficacy of Supplemental Instruction in Decoding Skills for Hispanic and Non-Hispanic Students in Early Elementary School,” 34 J. Special Ed. 90 (2000);B. Gunn et al., “Supplemental Instruction in Decoding Skills for Hispanic and Non-Hispanic Students in Early Elementary School: A Follow-Up,” 36 J. Special Ed. 69 (2002); Angela M. Przychodzin, “The Research Base for Direct Instruction in Mathematics,” SRA/McGraw-Hill, https://www.sraonline.com/download/ DI/Research/Mathematics/research_base_for%20di_math.pdf.
The Arundel DI experiment: “A Direct Challenge,” Ed. Week, Mar. 17, 1999, http://www.zigsite.com/DirectChallenge.htm; Martin A. Kozloff et al., Direct Instruction in Education, Jan. 1999, http://people.uncw.edu/kozloffm/diarti cle.htm; Nat’l Inst. Direct Instruction, http://www.nifdi.org; Daniel Radosh, “The Pet Goat Approach,” New Yorker, Jul. 26, 2004.
The Michigan study: L. Schweinhart et al., “Child-Initiated Activities in Early Childhood Programs May Help Prevent Delinquency,” 1 Early Child. Res. Q. 303–312 (1986). A larger proportion of the students who were taught using DI were male and were more likely to stay in state. These students might have been arrested more often in Michigan not because of DI but because men are more likely to commit crime and because people who were arrested out of state were not considered in the analysis. See also Paulette E. Mills et al., “Early Exposure to Direct Instruction and Subsequent Juvenile Delinquency: A Prospective Examination,” 69 Exceptional Child, 85–96 (2002) (finding no impact of Direct Instruction on subsequent juvenile delinquency).
The Bush administration approach: Daniel Radosh, “The Pet Goat Approach,” New Yorker, Jul. 26, 2004; U.S. Dep’t of Ed., What Works Clearinghouse, http://www.whatworks.ed.gov/; Southwest Ed. Dev. Lab., “What Does a Balanced Approach Mean?” http://www.sedl.org/reading/topics/bal anced.htm.
“Scientifically based” programs: The term was first found in the 1990s, promoting funding for “scientifically based” education research.
The president’s support of DI: In “The Pet Goat Approach,” Daniel Radosh speculated about a less pristine motive for our president’s support of the Direct Instruction curriculum, which is published by McGraw-Hill: “[I]t’s easy to imagine
one of [Michael] Moore’s hallmark montages, spinning circumstantial evidence into a conspirational web: a sepia-toned photograph from the thirties of, say, Prescott Bush and James McGraw, Jr., palling around on Florida’s Jupiter Island; a film clip from the eighties of Harold McGraw, Jr., joining the advisory panel of Barbara Bush’s literacy foundation; Harold McGraw III posing with President George W. Bush as part of his transition team; and, to tie it all together, former McGraw-Hill executive vice-president John Negroponte being sworn in as the new Ambassador to Iraq.” Daniel Radosh, “The Pet Goat Approach,” New Yorker, Jul. 26, 2004.
Loan decisions by the numbers: Peter Chalos, “The Superior Performance of Loan Review Committee,” 68 J. Comm. Bank Lending 60 (1985).
Disabling discretion: The value of disabling discretion isn’t just for the other guy. When we make snap judgments, we sometimes lose control of unconscious influences that can seep into our decision-making process. See generally Malcolm Gladwell, Blink: The Power of Thinking Without Thinking (2005). A few years ago, I collected information on how people tipped cab drivers in New Haven. Ian Ayres et al., “To Insure Prejudice: Racial Disparities in Taxicab Tipping,” 114 Yale L. J. 1613 (2005). A disheartening finding was that passengers tipped minority drivers about a third less than white drivers providing the same-quality service. At first, I was pretty sure that I didn’t tip minorities less because with very rare exceptions I always tip 20 percent. When I took a closer look at the data, however, I found that a lot of the passenger discrimination came from a seemingly innocuous source. Cab passengers like to round the total amount that they pay to a round number. We’ve all had the experience. Just before your cab arrives the fare clicks over to $7. Think fast. Do you leave $8 or $9? We suspect that passengers are often called upon to make a quick decision about whether to round up or down. Even people who think that they are hard-wired 20 percent tippers may find that unconscious factors influence this rounding decision. In my study, passengers were much more likely to round up with white drivers and more likely to round down with minority drivers. Suddenly, I’m not so confident that “I certainly don’t discriminate.” I’m more of a discretionary tipper than I had ever imagined.