The Undoing Project

by Michael Lewis

  Israel was still less a nation than a fort, and yet its army was in a state of barely controlled chaos. The soldiers were poorly trained, the units poorly coordinated. The head of the tank division didn’t even speak the same language as most of his men. In the early 1950s there was no formal war between Arabs and Jews, but the senseless metronomic violence exposed vulnerabilities in the Israeli military. The soldiers tended to cut and run at the first sign of trouble, for instance; and the officers tended to lead from behind. The infantry staged a succession of failed night raids on Arab outposts, during which Israeli troops got lost in the dark and never reached their targets. In one case, after a unit sent out to stage an attack spent the night wandering around in circles, the platoon commander had simply shot himself. When they managed to engage the enemy, the results were often disastrous. In October 1953, an Israeli unit that may or may not have been given instructions not to harm civilians had raided a Jordanian village and killed sixty-nine people, half of them women and children.

  Since the First World War, the job of assessing and sorting young conscripts into armies had fallen to psychologists, mainly because some ambitious psychologists had talked the U.S. Army into giving them the job. Still, if you need to sort tens of thousands of young men quickly into an efficient fighting force, it’s not immediately obvious that you also need a psychologist, and even less obvious when the only psychologist at hand is a twenty-one-year-old graduate of a two-year program who has more or less taught himself. Danny himself was surprised they asked him to do it, and did not feel equipped for the job. And he’d already seen the difficulty of trying to figure out which person was suited to which job when his superiors had asked him to evaluate candidates for officer training school.

  The young men applying to become officers had been given a weirdly artificial task: to move themselves from one side of a wall to the other without touching the wall, using only a long log that was not permitted to touch either the wall or the ground. “We noted who took charge, who tried to lead and was rebuffed, how cooperative each soldier was in contributing to the group effort,” Danny wrote. “We saw who seemed to be stubborn, submissive, arrogant, patient, hot-tempered, persistent, or a quitter. We saw competitive spite when someone whose idea had been rejected by the group sabotaged its efforts. And we saw reactions to crisis. . . . Under the stress of the event, we felt, each man’s true nature was revealed. The impression we had of each candidate’s character was as direct and compelling as the color of the sky.”

  He had had no trouble identifying which men would make good officers and which would not. “We were quite willing to declare, ‘This one will never make it,’ ‘That fellow is rather mediocre,’ or ‘He will be a star.’” The problem came when he’d tested his predictions against the outcomes—how the various candidates had actually performed in officer training. His predictions were worthless. And yet, because it was the army and he had a job to do, he kept on making them; and because he was Danny, he noted that he still felt confident about them. The situation reminded him of the famous Müller-Lyer optical illusion.

  Figure 2. Müller-Lyer optical illusion.

  Presented with two lines of equal length, the eye is tricked into seeing one as being longer than the other. Even after you prove to people, with a ruler, that the lines are identical, the illusion persists: They’ll insist that one line still looks longer than the other. If perception had the power to overwhelm reality in such a simple case, how much power might it have in a more complicated one?

  Danny’s commanding officers believed that each branch of the Israel Defense Forces had its own personality. There was a “fighter pilot” type, and an “armored unit” type, and an “infantry soldier” type, and so on. They wanted Danny to determine for which branch any particular recruit was best suited. Danny set out to create a personality test that would effectively sort the entire population of Israel into the correct buckets. He began by listing the handful of traits he thought most obviously correlated with a man’s fitness for combat service: masculine pride, punctuality, sociability, sense of duty, capacity for independent thought. “The list of traits was not derived from anything,” he later said. “I just thought it up. A professional would take years to do it, using pre-tests, trying out multiple versions, etcetera, but I didn’t know it was difficult to do.”

  The hard part, Danny thought, was getting an accurate measure of any of these traits from an ordinary job interview. The subtle difficulties that arise when people evaluate other people had been described back in 1915 by an American psychologist named Edward Thorndike. Thorndike asked U.S. Army officers to rate their men according to some physical trait (“physique,” for example) and then assess some less tangible quality (“intelligence,” “leadership,” and so forth). He discovered that the feeling created by making the first ranking bled into the second: If an officer thought a soldier physically impressive, he also found him impressive in other ways. Switch the order of assessment, and the same problem occurred: If a person was first judged to be generally great, he was then judged to be stronger than he actually was. “Obviously a halo of general merit is extended to influence the rating for the special ability, or vice versa,” Thorndike concluded; he went on to say that he had “become convinced that even a very capable foreman, employer, teacher, or department head is unable to view an individual as a compound of separate qualities and to assign a magnitude to each of these in independence of the others.” Thus was born what is still called “the halo effect.”

  Danny knew of the halo effect. And he could see that the Israeli army interviewers had been its victims: They had been spending twenty minutes with each new recruit and from the encounter offering a general impression of the recruit’s character. General impressions had been proven to be misleading, and so Danny wanted to avoid them. For that matter, he wanted to avoid having to rely on human judgment. Exactly why he mistrusted human judgment he was unsure. In retrospect, he suspected he must have read a recent book by Paul Meehl—the same Meehl who wondered what, if anything, unified the field of psychology. Meehl’s book, called Clinical versus Statistical Prediction, had shown that psychoanalysts who tried to predict what would become of their neurotic patients fared poorly compared to simple algorithms. Published in 1954—just a year before Danny overhauled the Israeli army’s assessment of the country’s youth—it had angered psychoanalysts, who believed that their clinical judgments and predictions had great value. It also raised a more general question: If these putative experts could be misled about the value of their predictions, who would not be misled? “All I know is that I must have read Meehl because of what I did,” said Danny.

  What he did was teach the army interviewers—young women, mainly—how to put a list of questions to each recruit to minimize the halo effect. He told them to pose very specific questions, designed to determine not how a person thought of himself but how the person had actually behaved. The questions were not just fact-seeking but designed to disguise the facts being sought. And at the end of each section, before moving on to the next, the interviewer was to assign a rating from 1 to 5 that corresponded with choices ranging from “never displays this kind of behavior” to “always displays this kind of behavior.” So, for example, when evaluating a recruit’s sociability, they’d give a 5 to a person who “forms close social relationships and identifies completely with the whole group” and a 1 to “a person who was “completely isolated.” Even Danny could see that there were all kinds of problems with his methods, but he didn’t have the time to worry too much about them. For instance, he briefly agonized over how to define a 3—was it someone who was extremely sociable on occasion, or someone who was moderately sociable all the time? Both, he basically decided. The big thing was that the judge was to keep her private opinions to herself. The question was not “What do I think of him?” but “What has he done?” The judgment of who went where in the Israeli army was to be made by Danny’s algorithm. “The interview
ers hated it,” he recalled. “I had a mutiny on my hands. I still remember one of them saying, ‘You’re turning us into robots.’ They had a sense that they could tell [a person’s character]. And I was robbing them of that. And they really didn’t like it.”

  Danny then had himself driven by an assistant around the country so that he could ask army officers to assign character trait ratings to their own soldiers—which he could then compare to the soldiers’ performance. Find the characteristics of the people who are good in a particular branch of the military, his thinking went, and you could use them to identify others who shared those traits and should be assigned to that branch. (His memory of his trip was typically unusual, preserving a curious detail rather than the broad picture. He didn’t recall much about his encounters with combat officers, but he remembered vividly what the driver had said after Danny had taken the wheel of the jeep. Danny had never before driven. After he braked in anticipation of a bump in the road, the driver praised him: “He said, ‘That is exactly the right gentleness.’”) From the combat officers in the field Danny learned that he’d been sent on a fool’s errand. The military stereotypes were false. There were no meaningful differences between the personalities of successful people in the different branches. The personality that succeeded in the infantry was more or less the same as the personality that succeeded beside an artillery piece or inside a tank.

  The scores on Danny’s personality test did predict something, however. They predicted the likelihood the recruit would succeed in any job. They gave the Israeli army a better idea than it had before of who would succeed as an officer, or as a member of some elite service (fighter pilot, paratrooper), and who would not. (They also turned out to predict who would end up in jail.) Maybe more surprisingly, the results were only loosely correlated with intelligence and education—which is to say they contained information that those simple measures did not. The effect of what became known informally as the “Kahneman score” was to make better military use of an entire nation and, in particular, to reduce, in the selection of its military leaders, the importance of raw, measurable intelligence and increase the importance of the qualities on Danny’s list.

  The process Danny created proved to be so successful that the Israeli military has used it right up to the present day with only minor adjustments. (When women were admitted to combat units, for instance, “masculine pride” became “pride.”) “They tried to really change it once,” says Reuven Gal, the author of A Portrait of the Israeli Soldier. Gal served for five years as chief psychologist of the Israel Defense Forces. “They made it worse, so they changed it back.” Upon leaving the army in 1983, Gal went to Washington, DC, on a National Academy of Sciences research associateship. There, one day, he had a call from a top general in the Pentagon. “He says, ‘Would you mind coming to talk to us?’” Gal went over to the Pentagon to be interrogated by a roomful of U.S. Army generals. They put their question in many different ways, but, Gal said, “It was always the same question: ‘Please explain to me how it is possible you guys use the same rifles we use, drive the same tanks we drive, fly the same airplanes we fly, and you are doing so well winning all of the battles and we are not? I know it’s not the weapons. It must be the psychology. How do you pick the soldiers for combat?’ For the next five hours they picked my brain about one thing: our selection process.”

  Later, when he was a university professor, Danny would tell students, “When someone says something, don’t ask yourself if it is true. Ask what it might be true of.” That was his intellectual instinct, his natural first step to the mental hoop: to take whatever someone had just said to him and try not to tear it down but to make sense of it. The question the Israeli military had asked him—Which personalities are best suited to which military roles?—had turned out to make no sense. And so Danny had gone and answered a different, more fruitful question: How do we prevent the intuition of interviewers from screwing up their assessment of army recruits? He’d been asked to divine the character of the nation’s youth. Instead he’d found out something about people who try to divine other people’s character: Remove their gut feelings, and their judgments improved. He’d been handed a narrow problem and discovered a broad truth. “The difference between Danny and the next nine hundred and ninety-nine thousand nine hundred and ninety-nine psychologists is his ability to find the phenomenon and then explain it in a way that applies to other situations,” said Dale Griffin, a psychologist at the University of British Columbia. “It looks like luck but he keeps doing it.”

  A different, more ordinary person would have left the experience brimming with confidence. In a stroke, twenty-one-year-old Danny Kahneman had exerted more influence upon the Israeli army—the institution on which the society depended for its survival—than any psychologist had ever done or ever would do. The obvious next step for him was to go off and get his PhD and become Israel’s leading expert in personality assessment and selection processes. Harvard was home to some of the leading figures in the field, but Danny decided, without anyone’s help, that he wasn’t bright enough to go to Harvard—and didn’t bother to apply. Instead he went to Berkeley.

  When he returned to Hebrew University as a young assistant professor in 1961, after four years away, he was freshly inspired by personality studies being done by the psychologist Walter Mischel. In the early 1960s Mischel created these wonderfully simple tests on children that wound up revealing a lot about them. In what became known as the “marshmallow experiment,” Mischel put three-, four-, and five-year-old kids in a room alone with their favorite treat—a pretzel stick, a marshmallow—and told them that if they could last a few minutes without eating the treat they’d receive a second treat. A small child’s ability to wait turned out to be correlated with his IQ and his family circumstances and some other things as well. Tracking the kids through life, Mischel later found that the better a five-year-old resisted the temptation, the higher his future SAT scores and his sense of self-worth, and the lower his body fat and the likelihood he’d suffer from some addiction.

  Gripped by a new enthusiasm, Danny designed a bunch of marshmallow test–like experiments. He even coined a phrase for what he was doing: the psychology of single questions. He arranged for Israeli kids on camping trips—this was just one example—to be offered a choice between sleeping in a single tent, a two-person tent, or an eight-person tent. Perhaps their answers, Danny thought, would say something about their tendency to affiliate with a group. The idea yielded either no findings or findings he couldn’t replicate in a subsequent experiment. And so he gave up. “I wanted to be a scientist,” he said. “And I thought, I can’t be a scientist unless I can replicate myself. I couldn’t replicate myself.” Doubting himself once again, he abandoned the study of personality, deciding he had no talent for it.

  * * *

  * Decades later, when Danny Kahneman was in his forties, he sat in for a day on a class at the University of California, Berkeley, taught by a psychologist named Eleanor Rosch. On that day, Rosch put a group of first-year graduate students through an exercise. She passed around a hat stuffed with slips of paper, on each slip a different occupation: zookeeper, airline pilot, carpenter, thief. The students were told to pick an occupation and then say what, if anything, popped to mind that foreshadowed their fate. Of course I wound up a zookeeper; as a kid I loved to cage our cat. The exercise was meant to illustrate the powerful instinct people have for finding causes for any effect, and also for creating narratives. “The whole group opens their papers at the same time,” recalled Rosch, “and within seconds someone laughs, and the laughter becomes general. And, yes, to their surprise, things have popped into their minds. Danny was the lone exception. “ ‘Nope,’ he said,” according to Rosch. “ ‘I could only have been two things. A psychologist or a rabbi.’”

  † The word is German and means “shape” or “form” but, in a manner the Gestalt psychologists would enjoy, has itself tended to change shape, depending on the conte
xt in which it is used.

  3

  THE INSIDER

  Amnon Rapoport was just eighteen years old when he was identified by the Israeli army’s new selection system as leadership material. They’d made him a tank commander. “I didn’t even know there was a tank corps,” he said. One night in October 1956 he drove his tank into Jordan to avenge the murder of several Israeli civilians. On these raids you never knew what decisions you might have to make quickly. Shoot or hold fire? Kill or let live? Live or die? A few months earlier, an Israeli soldier Amnon’s age had been captured by the Syrians. He’d decided to kill himself before they could question him. When the Syrians sent his body back, the Israeli army found a note in his toenail: “I never betrayed.”

  On that night in October 1956, Amnon’s first decision had been to stop firing: His job was to bombard the second floor of a Jordanian police building until Israeli paratroopers stormed the ground floor. He worried about killing his own men. After he’d stopped shelling he heard, over his tank’s radio, reports from the ground. “And all of a sudden, the reality hit me; this was not just an adventure with heroes and villains acting their role. People were dying.” The paratroopers were Israel’s elite fighting force. Their unit, in hand-to-hand combat, was suffering serious causalities, and yet their reports from the battle to Amnon’s ears inside the tank sounded calm, almost casual. “There was no panic,” he said, “indeed, no change of intonation and hardly any expression of emotion.” These Jews had become Spartans: How had that happened? He wondered how he would fare in hand-to-hand combat. He aspired to be a warrior, too.

  Two weeks later he drove his tank into Egypt, in what turned out to be the start of a military invasion. In the fog of battle, he was strafed not just by Egyptian but also Israeli warplanes. His most vivid memory was of an Egyptian MiG-15 diving straight down on his tank while he—with his head above the turret to maintain a 360-degree view of the battlefield—shouted at his driver to zig and zag to avoid being hit. It felt like the MiG was on a special assignment to blow off his head. A few days later, desperate Egyptian soldiers in full retreat approached Amnon’s tank with their arms in the air. They begged for water and protection from the Bedouins who hunted them for their rifles and boots. The day before, he was murdering these people; now all he felt toward them was pity. He marveled again—“at how easy it is to shift from an efficient killing machine to compassionate human being, and how quick the switch may be.” How did that happen?