The Undoing Project

by Michael Lewis

  His first mistake, he decided, was to have paid insufficient attention to Joey Dorsey’s age. “He was insanely old,” says Morey. “He was twenty-four years old when we drafted him.” Dorsey’s college career was impressive because he was so much older than the people he played against. He’d been, in effect, beating up on little kids. Raising the weight the model placed on a player’s age flagged Dorsey as a weak NBA prospect; more tellingly, it improved the model’s judgments about nearly all of the players in the database. For that matter, Morey realized, there existed an entire class of college basketball player who played far better against weak opponents than against strong ones. Basketball bullies. The model could account for that, too, by assigning greater weight to games played against strong opponents than against weak ones. That also improved the model.

  Morey could see—or thought he could see—how the model had been fooled by Joey Dorsey. Its blindness to the value of DeAndre Jordan was far more troubling. The kid had played a single year of college basketball, not very effectively. It turned out that he had been a sensational high school player, had hated his college coach, and didn’t even want to be in school. How could any model predict the future of a player who had intentionally failed? It was impossible to see Jordan’s future in his college stats, and, at the time, there were no useful high school basketball statistics. So long as it relied almost exclusively on performance statistics, the model would always miss DeAndre Jordan. The only way to see him, it seemed, was with the eyes of an old-fashioned basketball expert. As it happens, Jordan had grown up in Houston under the eyes of Rockets scouts, and one of those scouts had wanted to draft him on the strength of what appeared to him undeniable physical talent. One of his scouts had seen what his model had missed!

  Morey—being Morey—had actually tested whether there were any patterns in the predictions made by his staff. He’d hired most of them and thought they were great, and yet there was no evidence any one of them was any better than the other, or the market, at predicting who would make it in the NBA and who would not. If there was any such thing as a basketball expert who could identify future NBA talent, he hadn’t found him. He certainly didn’t think that he was one. “Weighting my personal intuition more heavily did not cross my mind,” he said. “I trust my gut very low. I just think there’s a lot of evidence that gut instincts aren’t very good.”

  In the end, he decided that the Rockets needed to reduce to data, and subject to analysis, a lot of stuff that had never before been seriously analyzed: physical traits. They needed to know not just how high a player jumped but how quickly he left the earth—how fast his muscles took him into the air. They needed to measure not just the speed of the player but the quickness of his first two steps. That is, they needed to be even more geeky than they already were. “When things go wrong, that’s what people do,” said Morey. “They go back to the habits that succeeded in the past. My thing was: Let’s go back to first principles. If these physical tools are going to matter, let’s test them more rigorously than they’ve ever been tested before. The weights we placed on production in college had to go down, and the weights we placed on raw physical abilities had to go up.”

  But once you started to talk about a guy’s body and what it might or might not be able to do on an NBA court, there was a limit to the usefulness of even the objective, measurable information. You needed, or seemed to need, experts to look at the tools in action and judge how well they would function playing a different game, against better competition. You needed scouts to rate a player’s ability to do the various things they knew were most important to do on a basketball court: shooting, finishing, getting to the rim, offensive rebounding, and so on. You needed experts. The limits of any model invited human judgment back into the decision-making process—whether it helped or not.

  And thus began a process of Morey trying as hard as he’d ever tried at anything in his life to blend subjective human judgment with his model. The trick wasn’t just to build a better model. It was to listen both to it and to the scouts at the same time. “You have to figure out what the model is good and bad at, and what humans are good and bad at,” said Morey. Humans sometimes had access to information that the model did not, for instance. Models were bad at knowing that DeAndre Jordan sucked his freshman year in college because he wasn’t trying. Humans were bad at . . . well, that was the subject Daryl Morey now needed to study more directly.

  Freshly exposed to the human mind, Morey couldn’t help but notice how strangely it operated. When it opened itself to information that might be useful in evaluating an amateur basketball player, it also opened itself to being fooled by the very illusions that had made the model such a valuable tool in the first place. For instance, in the 2007 draft there had been a player his model really liked: Marc Gasol. Gasol was twenty-two years old, a seven-foot-one center playing in Europe. The scouts had found a photograph of him shirtless. He was pudgy and baby-faced and had these jiggly pecs. The Rockets staff had given Marc Gasol a nickname: Man Boobs. Man Boobs this and Man Boobs that. “That was my first draft in charge and I wasn’t so brave,” said Morey. He allowed the general ridicule of Marc Gasol’s body to drown out his model’s optimism about Gasol’s basketball future, and so instead of arguing with his staff, he watched the Memphis Grizzlies take Gasol with the 48th pick of the draft. The odds of getting an All-Star with the 48th pick in the draft were well below one in a hundred. The 48th pick of the draft basically never even yielded a useful NBA bench player, but already Marc Gasol was proving to be a giant exception.§ The label they’d stuck on him clearly had affected how they valued him: names mattered. “I made a new rule right then,” said Morey. “I banned nicknames.”

  * * *

  All of a sudden he was right back in the mess he and his model had been hired to eliminate. If he could never completely remove the human mind from his decision-making process, Daryl Morey had at least to be alive to its vulnerabilities. He now saw these everywhere he turned. One example: Before the draft, the Rockets would bring a player in with other players and put him through his paces on the court. How could you deny yourself the chance to watch him play? But while it was interesting for his talent evaluators to see a player in action, it was also, Morey began to realize, risky. A great shooter might have an off day; a great rebounder might get pushed around. If you were going to let everyone watch and judge, you also had to teach them not to place too much weight on what they were seeing. (Then why were they watching in the first place?) If a guy was a 90 percent free-throw shooter in college, for instance, it really didn’t matter if he missed six free throws in a row during the private workout.

  Morey leaned on his staff to pay attention to the workouts but not allow whatever they saw to replace what they knew to be true. Still, a lot of people found it very hard to ignore the evidence of their own eyes. A few found the effort almost painful, as if they were being strapped to the mast to listen to the Sirens’ song. One day a scout came to Morey and said, “Daryl, I’ve done this long enough. I think we should stop having these workouts. Please, just stop doing them.” Morey said, Just try to keep what you are seeing in perspective. Just weight it really low. “And he says, ‘Daryl, I just can’t do it.’ It’s like a guy addicted to crack,” Morey said. “He can’t even get near it without it hurting him.”

  Soon Morey noticed something else: A scout watching a player tended to form a near-instant impression, around which all other data tended to organize itself. “Confirmation bias,” he’d heard this called. The human mind was just bad at seeing things it did not expect to see, and a bit too eager to see what it expected to see. “Confirmation bias is the most insidious because you don’t even realize it is happening,” he said. A scout would settle on an opinion about a player and then arrange the evidence to support that opinion. “The classic thing,” said Morey, “and this happens all the time with guys: If you don’t like a prospect, you say he has no position. If you like him, you say he’s multi
positional. If you like a player, you compare his body to someone good. If you don’t like him, you compare him to someone who sucks.” Whatever prejudice a person brought to the business of selecting amateur players he tended to preserve, even when it served him badly, because he was always looking to have that prejudice confirmed. The problem was magnified by the tendency of talent evaluators—Morey included—to favor players who reminded them of their younger selves. “My playing career is so irrelevant to my career,” he said. “And still I like guys who beat the shit out of people and cheat the rules and are nasty. Bill Laimbeer types. Because that’s how I played.” You saw someone who reminded you of you, and then you looked for the reasons why you liked him.

  The mere fact that a player physically resembled some currently successful player could be misleading. A decade ago a six-foot-two-inch, light-skinned, mixed-race guy who had gone unnoticed by major colleges in high school and so played for some obscure tiny college, and whose main talent was long-range shooting, would have had no obvious appeal. The type didn’t exist in the NBA—at least not as a raging success. Then Stephen Curry came along and set the NBA on fire, led the Golden State Warriors to an NBA championship, and was everyone’s most valuable player. Suddenly—just like that—all these sharp-shooting mixed-race guards were turning up for NBA job interviews and claiming that their game was a lot like Stephen Curry’s; and they were more likely to get drafted because of the resemblance.¶ “For five years after we drafted Aaron Brooks, we saw so many kids who compared themselves to Aaron. Because there are so many little guards.” Morey’s solution was to forbid all intraracial comparison. “We’ve said, ‘If you want to compare this player to another player, you can only do it if they are a different race.’” If the player in question was African American, for instance, the talent evaluator was only allowed to argue that “he is like so-and-so” if so-and-so was white or Asian or Hispanic or Inuit or anything other than black. A funny thing happened when you forced people to cross racial lines in their minds: They ceased to see analogies. Their minds resisted the leap. “You just don’t see it,” said Morey.

  Maybe the mind’s best trick of all was to lead its owner to a feeling of certainty about inherently uncertain things. Over and again in the draft you saw these crystal-clear pictures form in the minds of basketball experts which later proved a mirage. The picture in virtually every professional basketball scout’s mind of Jeremy Lin, for instance. The now world-famous Chinese American shooting guard graduated from Harvard in 2010 and entered the NBA draft. “He lit up our model,” said Morey. “Our model said take him with, like, the 15th pick in the draft.” The objective measurement of Jeremy Lin didn’t square with what the experts saw when they watched him play: a not terribly athletic Asian kid. Morey hadn’t completely trusted his model—and so had chickened out and not drafted Lin. A year after the Houston Rockets failed to draft Jeremy Lin, they began to measure the speed of a player’s first two steps: Jeremy Lin had the quickest first move of any player measured. He was explosive and was able to change direction far more quickly than most NBA players. “He’s incredibly athletic,” said Morey. “But the reality is that every fucking person, including me, thought he was unathletic. And I can’t think of any reason for it other than he was Asian.”

  In some strange way people, at least when they were judging other people, saw what they expected to see and were slow to see what they hadn’t seen before. How bad was the problem? When Jeremy Lin’s coach at the New York Knicks finally put him in the game—because everyone else was injured—and allowed him to light up Madison Square Garden, the Knicks were preparing to release Jeremy Lin. Jeremy Lin had already decided that if he was released he’d simply quit basketball altogether. That’s how bad the problem was: that a very good NBA player would never have been given a serious chance to play in the NBA, simply because the minds of experts had concluded he did not belong. How many other Jeremy Lins were out there?

  After the Houston Rockets and everyone else in the NBA neglected to see Jeremy Lin’s value in the draft (he signed after the draft as a free agent), the league shut down. A dispute between players and owners led to a lockout, and no one was allowed to work. Morey enrolled in an executive education course at Harvard Business School and took a class in behavioral economics. He’d heard of the discipline (“I’m not an idiot”) but had never studied it. At the start of the first class, the professor asked him and everyone else in the class to write down the last two digits of their cell phone on a sheet of paper. Then she asked the class to write down their best estimate of the number of African countries in the United Nations. Then she collected all the papers and showed them that the people whose cell phone numbers were higher offered systematically higher estimates of African countries in the United Nations. Then she took another example and said, “I’m going to do it again. I’m about to anchor you. Here. See if you aren’t screwed up.” Everyone had been warned; everyone’s minds remained screwed up. Simply knowing about a bias wasn’t sufficient to overcome it: The thought of that made Daryl Morey uneasy.

  When the NBA returned to work he made yet another unsettling discovery. Just before the draft, the Toronto Raptors called and offered to trade their high first-round draft pick for Houston’s backup point guard, Kyle Lowry. Morey talked about it with his staff, and they were on the brink of not doing the deal when one of the Rockets executives said, “You know, if we had the pick we’re thinking of trading for and they offered Lowry for it, we wouldn’t even consider it as a possibility.” They stopped and analyzed the situation more closely: The expected value of the draft pick exceeded, by a large margin, the value they placed on the player they’d be giving up for it. The mere fact that they owned Kyle Lowry appeared to have distorted their judgment about him.** Looking back over the previous five years, they now saw that they’d systematically overvalued their own players whenever another team tried to trade for them. Especially when offered the chance to trade one of their NBA players for another team’s draft picks, they’d refused deals they should have done. Why? They hadn’t done it consciously.

  Morey thus became aware of what behavioral economists had labeled “the endowment effect.” To combat the endowment effect, he forced his scouts and his model to establish, going into the draft, the draft pick value of each of their own players.

  The next season, before the trade deadline, Morey got up before his staff and listed on a whiteboard all the biases he feared might distort their judgment: the endowment effect, confirmation bias, and others. There was what people called “present bias”—the tendency, when making a decision, to undervalue the future in relation to the present. There was “hindsight bias”—which he thought of as the tendency for people to look at some outcome and assume it was predictable all along. The model was an antidote to these vagaries of human judgment, but, by 2012, the model seemed to be approaching a limit to the informational edge it would give the Rockets in valuing players. “Every year we talk about what to take out and what to put in the model,” said Morey. “And every year it gets a little more depressing.”

  This job of running a professional basketball team had turned out to be a bit different than he had imagined, back when he was a kid. It was as if he had been assigned to take apart a fiendishly complicated alarm clock to see why it wasn’t working, only to discover that an important part of the clock was inside his own mind.

  * * *

  Morey and his staff had obviously seen a lot of big men. But in the winter of 2015, even they were shocked by the sight of the Indian who walked into their interview room. He was dressed simply in sweatpants and a lime-green Nike T-shirt, with a pair of dog tags dangling from his neck. That neck—like his hands, his feet, his head, and even his ears—was so cartoonishly immense that you found your eyes jumping from feature to feature and wondering if that specific body part broke a Guinness book record. The Rockets once employed a seven-foot-six-inch Chinese center named Yao Ming whose size provoked these weir
d reactions in others. People would see him and turn and run, or burst out laughing, or weep. From head to toe the Indian was a few inches shorter than Yao Ming, but in every other way he was bigger. After seeing his measurements, and finding it hard to believe anyone could grow so much in just nineteen years, Morey had asked his staff to dig out his birth certificate. The Indian’s agent had come back and said that the village in which he’d been born kept no birth records. Hearing this, Morey recalled what Dikembe Mutombo had once told him. Mutombo was a seven-foot-two-inch shot blocker who had come to the Rockets by way of Congo, with stops in between at five other NBA teams. He said that whenever some huge guy from overseas turned up claiming to be a lot younger than he looked, “You need to cut open his legs and count the rings.”

  The Indian’s name was Satnam Singh. In all but his size he seemed young. He had the social uncertainty of an adolescent confused to find himself suddenly so far away from home. He smiled nervously and lowered himself into the chair at the head of the table.

  “You doing all right?” said the Rockets interviewer.

  “Yeah, I’m good good good.” It wasn’t a voice but a foghorn. So guttural it took a moment to work out what he’d said.

  “We just want to get to know you a bit better,” said the interviewer. “Tell us about your agent and why you selected him.”

  Satnam Singh rambled on nervously for a couple of minutes. It was unclear whether anyone in the room followed what he said. They gathered that he’d basically been taken care of since he was fourteen by people who imagined an NBA career for him.

  “Tell us about where you are from and your family?” the interviewer asked.