Beyond that, Amos was the most terrifying mind most people had ever encountered. “People were afraid to discuss ideas in front of him,” said a friend—because they were afraid he would put his finger on the flaw that they had only dimly sensed. One of Amos’s graduate students, Ruma Falk, said she was so afraid of what Amos would think of her driving that when she drove him home, in her car, she insisted that he drive. And now here he was spending all of his time with Danny, whose susceptibility to criticism was so extreme that a single remark from a misguided student sent him down a long, dark tunnel of self-doubt. It was as if you had dropped a white mouse into a cage with a python and come back later and found the mouse talking and the python curled in the corner, rapt.
But there was another story to be told, about how much Danny and Amos had in common. Both were grandsons of Eastern European rabbis, for a start. Both were explicitly interested in how people functioned when they were in a “normal” unemotional state. Both wanted to do science. Both wanted to search for simple, powerful truths. As complicated as Danny might have been, he still longed to do “the psychology of single questions,” and as complicated as Amos’s work might have seemed, his instinct was to cut through endless bullshit to the simple nub of any matter. Both men were blessed with shockingly fertile minds. And both were Jews, in Israel, who did not believe in God. And yet all anyone saw were their differences.
The most succinct physical manifestation of the deep difference between the two men was the state of their offices. “Danny’s office was such a mess,” recalled Daniela Gordon, who had become Danny’s teaching assistant. “Scraps on which he’d scribbled a sentence or two. Paper everywhere. Books everywhere. Books opened to places he’d stopped reading. I once found my master’s thesis open on page thirteen—I think that’s where he stopped. And then you would walk down the hall three or four rooms, and you come to Amos’s office . . . and there is nothing in it. A pencil on a desk. In Danny’s office you couldn’t find anything because it was such a mess. In Amos’s office you couldn’t find anything because there was nothing there.” All around them people watched and wondered: Why were they getting along so well? “Danny was a high-maintenance person,” said one colleague. “Amos was the last one to put up with a high-maintenance person. And yet he was willing to go along. Which was amazing.”
Danny and Amos didn’t talk much about what they got up to when they were alone together, which just made everyone else more curious about what it was. In the beginning they were kicking around Danny’s proposition—that people weren’t Bayesians, or conservative Bayesians, or statisticians of any sort. Whatever human beings did when presented with a problem that had a statistically correct answer, it wasn’t statistics. But how did you sell that to an audience of professional social scientists who were more or less blinded by theory? And how did you test it? They decided, in essence, to invent an unusual statistics test and give it to the scientists, and see how they performed. Their case would be built from evidence that consisted entirely of answers to questions they’d put to some audience—in this case, an audience of people trained in statistics and probability theory. Danny dreamed up most of the questions, many of which were sophisticated versions of the questions about red and white poker chips:
The mean IQ of the population of eighth graders in a city is known to be 100. You have selected a random sample of 50 children for a study of educational achievement. The first child tested has an IQ of 150. What do you expect the mean IQ to be for the whole sample?
At the end of the summer of 1969, Amos took Danny’s questions to the annual meeting of the American Psychological Association, in Washington, DC, and then on to a conference of mathematical psychologists. There he gave the test to roomfuls of people whose careers required fluency in statistics. Two of the test takers had written statistics textbooks. Amos then collected the completed tests and flew home with them to Jerusalem.
There he and Danny sat down to write together for the first time. Their offices were tiny, so they worked in a small seminar room. Amos didn’t know how to type, and Danny didn’t particularly want to, so they sat with notepads. They went over each sentence time and again and wrote, at most, a paragraph or two each day. “I had this sense of realization: Ah, this is not going to be the usual thing, this is going to be something else,” said Danny. “Because it was funny.”
When Danny looked back on that time, what he recalled mainly was the laughter—what people outside heard from the seminar room. “I have the image of balancing precariously on the back legs of a chair and laughing so hard I nearly fell backwards.” The laughter might have sounded a bit louder when the joke had come from Amos, but that was only because Amos had a habit of laughing at his own jokes. (“He was so funny that it was okay he was laughing at his own jokes.”) In Amos’s company Danny felt funny, too—and he’d never felt that way before. In Danny’s company Amos, too, became a different person: uncritical. Or, at least, uncritical of whatever came from Danny. He didn’t even poke fun in jest. He enabled Danny to feel, in a way he hadn’t before, confident. Maybe for the first time in his life Danny was playing offense. “Amos did not write in a defensive crouch,” he said. “There was something liberating about the arrogance—it was extremely rewarding to feel like Amos, smarter than almost everyone.” The finished paper dripped with Amos’s self-assurance, beginning with the title he had put on it: “Belief in the Law of Small Numbers.” And yet the collaboration was so complete that neither of them felt comfortable taking the credit as the lead author; to decide whose name would appear first, they flipped a coin. Amos won.
“Belief in the Law of Small Numbers” teased out the implications of a single mental error that people commonly made—even when those people were trained statisticians. People mistook even a very small part of a thing for the whole. Even statisticians tended to leap to conclusions from inconclusively small amounts of evidence. They did this, Amos and Danny argued, because they believed—even if they did not acknowledge the belief—that any given sample of a large population was more representative of that population than it actually was.
The power of the belief could be seen in the way people thought of totally random patterns—like, say, those created by a flipped coin. People knew that a flipped coin was equally likely to come up heads as it was tails. But they also thought that the tendency for a coin flipped a great many times to land on heads half the time would express itself if it were flipped only a few times—an error known as “the gambler’s fallacy.” People seemed to believe that if a flipped coin landed on heads a few times in a row it was more likely, on the next flip, to land on tails—as if the coin itself could even things out. “Even the fairest coin, however, given the limitations of its memory and moral sense, cannot be as fair as the gambler expects it to be,” they wrote. In an academic journal that line counted as a splendid joke.
They then went on to show that trained scientists—experimental psychologists—were prone to the same mental error. For instance, the psychologists who were asked to guess the mean IQ of the sample of kids, in which the first kid was found to have an IQ of 150, often guessed that it was 100, or the mean of the larger population of eight graders. They assumed that the kid with the high IQ was an outlier who would be offset by an outlier with an extremely low IQ—that every heads would be followed by a tails. But the correct answer—as produced by Bayes’s theorem—was 101.
Even people trained in statistics and probability theory failed to intuit how much more variable a small sample could be than the general population—and that the smaller the sample, the lower the likelihood that it would mirror the broader population. They assumed that the sample would correct itself until it mirrored the population from which it was drawn. In very large populations, the law of large numbers did indeed guarantee this result. If you flipped a coin a thousand times, you were more likely to end up with heads or tails roughly half the time than if you flipped it ten times. For some reason huma
n beings did not see it that way. “People’s intuitions about random sampling appear to satisfy the law of small numbers, which asserts that the law of large numbers applies to small numbers as well,” Danny and Amos wrote.
This failure of human intuition had all sorts of implications for how people moved through the world, and rendered judgments and made decisions, but Danny and Amos’s paper—eventually published in the Psychological Bulletin—dwelled on its consequences for social science. Social science experiments usually involved taking some small sample from a large population and testing some theory on it. Say a psychologist thought that he had discovered a connection: Children who preferred to sleep alone on camping trips were somewhat less likely to participate in social activities than were children who preferred eight-person tents. The psychologist had tested a group of twenty kids, and they confirmed his hypothesis. Not every child who wanted to sleep alone was asocial, and not every child who longed for an eight-person tent was highly sociable—but the pattern existed. The psychologist, being a conscientious scientist, selects a second sample of kids—to see if he can replicate this finding. But because he has misjudged how large the sample needs to be if it is to stand a good chance of reflecting the entire population, he is at the mercy of luck.* Given the inherent variability of the small sample, the kids in his second sample might be unrepresentative, not at all like most children. And yet he treated them as if they had the power to confirm or refute his hypothesis.
The belief in the law of small numbers: Here was the intellectual error that Danny and Amos suspected that a lot of psychologists made, because Danny had made it. And Danny had a far better feel for statistics than most psychologists, or even most statisticians. The entire project, in other words, was rooted in Danny’s doubts about his own work, and his willingness, which was almost an eagerness, to find error in that work. In their joint hands, Danny’s tendency to look for his own mistakes became the most fantastic material. For it wasn’t just Danny who made those mistakes: Everyone did. It wasn’t just a personal problem; it was a glitch in human nature. At least that was their suspicion.
The test they administered to psychologists confirmed that suspicion. When seeking to determine if the bag they held contained mostly red chips, psychologists were inclined to draw, from very few chips, broad conclusions. In their search for scientific truth, they were relying far more than they knew on chance. What’s more, because they had so much faith in the power of small samples, they tended to rationalize whatever they found in them.
The test Amos and Danny had created asked the psychologists how they would advise a student who was testing a psychological theory—say, that people with long noses are more likely to lie. What should the student do if his theory tests as true on one sample of humanity but as false on another? The question Danny and Amos put to the professional psychologists was multiple-choice. Three of the choices involved telling the student either to increase his sample size or, at the very least, to be more circumspect about his theory. Overwhelmingly, the psychologists had plunked for the fourth option, which read: “He should try to find an explanation for the differences between the two groups.”
That is, he should seek to rationalize why in one group people with long noses are more likely to lie, while in the other they are not. The psychologists had so much faith in small samples that they assumed that whatever had been learned from either group must be generally true, even if one lesson seemed to contradict the other. The experimental psychologist “rarely attributes a deviation of results from expectations to sampling variability because he finds a causal ‘explanation’ for any discrepancy,” wrote Danny and Amos. “Thus, he has little opportunity to recognize sampling variation in action. His belief in the law of small numbers, therefore, will forever remain intact.”
To which Amos, by himself, appended: “Edwards . . . has argued that people fail to extract sufficient information or certainty from probabilistic data; he called this failure conservatism. Our respondents can hardly be described as conservative. Rather, in accord with the representation hypothesis, they tend to extract more certainty from the data than the data, in fact, contain.” (“Ward Edwards was established,” said Danny. “And we were taking pot shots—Amos was sticking his tongue out at him.”)
By the time they were finished with the paper, in early 1970, they had lost any clear sense of their individual contributions. It was nearly impossible to say, of any given passage, whether more of some idea had come from Danny or from Amos. Far more easily determined, at least for Danny, was responsibility for the paper’s confident, almost brazen, tone. Danny had always been a nervous scholar. “If I had written it alone, in addition to being tentative and having a hundred references, I would probably have confessed that I am only a recently reformed idiot,” he said. “I could have done the paper all by myself. Except that if I had done it alone people would not have paid it attention. It had a star quality. And I attributed that quality to Amos.”
He thought that their paper was funny and provocative and interesting and arrogant in a way he could never be on his own, but in truth he didn’t think any more than that—and he didn’t think Amos did, either. Then they gave the paper to a person they assumed would be a skeptical audience, a psychology professor at the University of Michigan named Dave Krantz. Krantz was a serious mathematician, and also one of Amos’s coauthors on the impenetrable multivolume Foundations of Measurement. “I thought it was a stroke of genius,” recalled Krantz. “I still think it is one of the most important papers that has ever been written. It was counter to all the work that was being done—which was governed by the idea that you were going to explain human judgment by correcting for some more or less minor error to the Bayesian model. It was exactly contrary to the ideas that I had. Statistics was the way you should think about probabilistic situations, but statistics was not the way people did it. Their subjects were all sophisticated in statistics—and even they got it wrong! Every question in the paper that the audience got wrong I felt the temptation to get wrong.”
That verdict—that Danny and Amos’s paper wasn’t just fun but important—would eventually be echoed outside of psychology. “Over and over again economists say, ‘If the evidence of the world tells you it is true, then people figure out what’s true,’” says Matthew Rabin, a professor of economics at Harvard University. “That people are, in effect, very good statisticians. And if they aren’t—well, they don’t survive. And so if you are going down the list of things that are important in the world, the fact that people don’t believe in statistics is pretty important.”
Danny, being Danny, was slow to accept the compliment. (“When Dave Krantz said, ‘It’s a breakthrough,’ I thought he was out of his mind.”) Still, he and Amos were onto something far bigger than an argument about how to use statistics. The power of the pull of a small amount of evidence was such that even those who knew they should resist it succumbed. People’s “intuitive expectations are governed by a consistent misperception of the world,” Danny and Amos had written in their final paragraph. The misperception was rooted in the human mind. If the mind, when it was making probabilistic judgments about an uncertain world, was not an intuitive statistician, what was it? If it wasn’t doing what the leading social scientists thought it did, and economic theory assumed that it did, what, exactly, was it doing?
* * *
* A lot of psychologists at the time, including Danny, were using sample sizes of 40 subjects, which gave them only a 50 percent chance of accurately reflecting the population. To have a 90 percent chance of capturing the traits of the larger population, the sample size needed to be at least 130. To gather a larger sample of course required a lot more work, and thus slowed a research career.
6
THE MIND’S RULES
In 1960 Paul Hoffman, a professor of psychology at the University of Oregon with a special interest in human judgment, persuaded the National Science Foundation to give him sixty thousand doll
ars so that he could quit his teaching job and create what he described as a “center for basic research in the behavioral sciences.” He’d never really enjoyed teaching all that much and was frustrated by how slowly academic life moved, especially in granting him promotions. And so he quit and bought a building in a leafy Eugene neighborhood that had most recently housed a Unitarian church, and renamed it the Oregon Research Institute. A private institution devoted exclusively to the study of human behavior, there was nothing in the world like it, and it soon attracted both curious assignments and unusual people. “Here brainy people, working in the proper atmosphere, go quietly about their task of finding out what makes us tick,” a local Eugene paper reported.
The vagueness of that account became typical of descriptions of the Oregon Research Institute. No one really knew what the psychologists inside were up to—only that they could no longer say “I’m a professor” and leave it at that. After Paul Slovic left the University of Michigan to join Hoffman in his new research center, and his small children asked him what he did for a living, he would point to a poster that depicted a brain sectioned into its various compartments and say, “I study the mysteries of the mind.”
The Undoing Project Page 15