Scorecasting: The Hidden Influences Behind How Sports Are Played and Games Are Won

by L. Jon Wertheim

  A SAMPLE OF THE POINT VALUES FROM MCCOY’S CHART

  According to the chart, the value of the first pick in the draft (3000) was equal to the combined value of the sixth pick (1600) and eighth pick (1400) but more than that of the final four picks of the first round (640 + 620 + 600 + 590 = 2450) combined.

  McCoy is quick to admit that it was a crass calculation, hardly built on rigorous econometrics. But armed with the chart, the Cowboys approached draft day in 1991 with supreme confidence. Whenever a trade offer came over the transom, they’d simply consult their conversion chart, make a few calculations, and determine whether it was worthwhile. If it was “below the line,” it was best to pass; above the line, it was probably a steal. After using the first pick of the first round to select Russell Maryland, a defensive tackle Johnson had once recruited to the University of Miami, Dallas traded two of its first-round selections. They wheeled, they dealed, they took 17 players in all (including three eventual Pro Bowl players); by the time the weekend was over, the Cowboys’ inner sanctum was drunk with confidence. Draining a few bottles of beer, Johnson told an embedded Sports Illustrated reporter, “We’ll be good; big-time good. There’s no doubt in anybody’s mind here.… I couldn’t care less what the people out there think of us.”

  Sure enough, in the ensuing years, the Cowboys gained back the aura of America’s Team. The Blue and Silver mystique returned as Dallas won three Super Bowls over the next five years.

  The franchise’s turnaround was due in no small part to the Cowboys’ exceptional success on draft day. In the five years after the unveiling of the chart, Dallas selected 15 starters and five Pro Bowl players. “It got to the point,” says McCoy, “where teams were afraid to trade with us.” Jerry Jones soon began referring to the chart as Dallas’s secret weapon.

  There were ancillary benefits as well. Using McCoy’s bible, the Cowboys were able to identify other teams that consistently overpaid for talent. “Those were the teams we wanted to call!” says McCoy. In 1999, for instance, the New Orleans Saints famously traded eight draft picks, including all their 1999 selections, to the Washington Redskins in order to draft Ricky Williams with the fifth pick in the first round. At least according to the values of the chart, New Orleans had overpaid to comical proportions, and this did not go unnoticed in the Cowboys’ war room.* Note to self: Trade with New Orleans whenever possible.

  It was probably inevitable, but the Cowboys’ secret weapon didn’t stay secret forever. With the Cowboys winning so prodigiously, it was only natural that their coaches and coordinators would attract the interest of other teams. Before Dave Wannstedt went to coach the Chicago Bears or Norv Turner took the head job with the Washington Redskins, they made sure to grab a copy of the franchise’s sacred text as they packed. Dallas scouts and front office employees also took the chart with them as they decamped for other teams. Within a decade, most, if not all, teams in the league had a purloined copy of McCoy’s creation.

  In 1996, Jones bought out his buddy McCoy, though to this day the two remain close friends and business partners in natural gas ventures. By then the Cowboys were worth $300 million, more than double Jones’s purchase price. (Today, Wall Street values the franchise at close to $2 billion—more than 12 times what Jones paid.) Now an investor in Dallas, McCoy chuckles when he considers the legacy of his creation. “I guess it leveled the playing field and made trading easier because everyone could point to the chart and cover their butt,” he said. Then he added forlornly, “But after a while, you couldn’t [fleece] other teams the way we used to.”

  Or could you? After all, McCoy’s creation was an artifact of what teams did, not necessarily what they should do. The chart provided the average value of draft picks based on actual trades teams made, and so the Cowboys could tell whether a certain trade was above or below the average value other NFL teams placed on those players. But what if the average value teams placed on draft picks was wrong? Sure, every team now had a copy of the chart, but few teams double-checked McCoy’s valuations or updated them in accordance with salary cap changes or, more important, the performance of the actual picks. Did anyone stop to check whether the number one pick really was more than twice as good as the number eight pick, as the chart dictated? No. “We’re football guys, not math majors,” said one executive. “We’re all using this document that a buddy of Jerry Jones put together using picks in, like, the late eighties? Now that you put it like that, it’s probably not so smart.”

  Definitely not so smart, at least according to two prominent behavioral economists who studied the NFL draft. In 2004, Richard Thaler, a professor of behavioral economics at the University of Chicago, and Cade Massey at Yale were watching the NFL draft. With the first pick, the San Diego Chargers chose quarterback Eli Manning, the brother of perhaps the best quarterback in the league, Peyton Manning, and the son of longtime NFL quarterback Archie Manning. The New York Giants held the number four pick and were in the market for a premier quarterback as well. It was no secret they coveted Manning and thought he was the best prospect.

  As the estimable Peter King from Sports Illustrated reported at the time, during the 15 minutes the Giants had to make their selection, they were ambushed with two very different options. Option 1 was to make a trade with San Diego in which the Giants would first draft Philip Rivers—considered the second-best quarterback prospect in the draft—and then swap him for Manning plus give up their third-round pick (number 65) that year as well as their first- and fifth-round picks in the 2005 draft. Option 2 was to trade down with the Cleveland Browns, who held the seventh pick and also wanted a quarterback. At number seven, the Giants probably would draft the consensus third-best quarterback in the draft, Ben Roethlisberger. In exchange for moving down, the Giants would also receive from Cleveland their second-round pick (number 37) that year.

  The Giants chose the first option, which meant they effectively considered Eli Manning to be worth more than Ben Roethlisberger plus four additional players. It turns out that this matched the chart perfectly.

  To the two economists, however, this seemed like an extraordinarily steep price. They wondered whether perhaps the circumstances were exceptional; perhaps Manning’s extraordinary pedigree reduced risk and made him a special case. But after collecting data from the NFL draft over the previous 13 years and looking at trades made on draft day as well as the compensation—salaries plus bonuses—paid to top picks, they found that the Manning trade was anything but unusual. As a matter of routine, if not rule, teams paid huge prices in terms of current and future picks to move up in the draft. They also paid dearly for contracts with those players.

  In Manning’s case, not only did he effectively cost the Giants four other players—one of whom turned out to be All-Pro linebacker Shawne Merriman—he was also given a six-year $54 million contract. Compare this to Roethlisberger, ultimately drafted eleventh by the Pittsburgh Steelers, who received $22.26 million over six years. Massey and Thaler found that historically, the number one pick in the draft typically is paid about 80 percent—80 percent!—more than the eleventh pick on the initial contract.

  Massey and Thaler also found that the inflated values teams were assigning to high picks were remarkably, if not unbelievably, consistent. From used cars to commodities to real estate, markets inevitably vary. After all, different people with different needs and different resources make different valuations, and you’d think the market for football players, inherently subjective and speculative, would be especially erratic. But when it came to NFL draft picks, virtually every team agreed on the same values. No matter the circumstances or a team’s needs, teams routinely assigned the same value to the same pick. Why?

  It turned out they were all using the chart Mike McCoy created in 1991!

  To test their suspicions that the chart overvalued high picks, Massey and Thaler compared the values teams placed on picks—either in terms of the picks and players they gave up or in terms of compensation—with the actual performance of the play
ers. The economists then compared those numbers with the performance of the players given up to get those picks. For example, in the case of Eli Manning, how did his performance over the next five years compare with that of Philip Rivers plus the performances of the players chosen with the picks the Giants had to give San Diego to get Manning? Likewise, how did those numbers stack up against Ben Roethlisberger’s stats and those of the players the Giants could have had with the additional picks they would have received from Cleveland?

  More generally, if the chart says the number one pick will cost you the number six pick plus the number eight pick, if the chart is right, the performance of the number one player drafted should be the same as the total performance of the number six and number eight picks combined. The economists looked at the probability of making the roster, the number of starts, and the likelihood of making the Pro Bowl.

  They found that higher picks are better than lower picks on average and that first rounders on average post better numbers than do second rounders, who in turn post better stats than third-round draft picks, and so on. No one will try to tell you that collectively first-round picks do not end up as better pros than third-round picks or that third-round picks don’t outperform sixth-round picks.

  The problem was that they weren’t that much better. For example, according to the chart, the number one pick in the first round should be worth roughly five times the thirty-third pick, that is, the first pick in the second round. But it turns out that the top pick on average is not even twice as good as the thirty-third-picked player, yet teams pay the number one pick four to five times more than the thirty-third player drafted. Even within the first round, the chart claims that the number ten pick is worth less than half as much as the number one pick and accordingly is paid about half as much. But in reality, the typical number ten pick is almost as good a player as the typical number one pick.

  Even looking position by position, the top draft picks are overvalued. How much better is the first quarterback or receiver taken than the second or third quarterback or receiver? Not much. The researchers concluded the following:

  The probability that the first player drafted at a given position is better than the second player drafted at the same position is only 53 percent, that is, slightly better than a tie.

  The probability that the first player drafted at a position is better than the third player drafted at the same position is only 55 percent.

  The probability that the first player drafted at a position is better than the fourth player drafted at the same position is only 56 percent.

  In other words, selecting the consensus top player at a specific position versus the consensus fourth-best player at that position increases performance, measured by the number of starts, by only 6 percent. And even this is overstating the case, since the number one pick is afforded more chances/more starts simply because the team has invested so much money in him. Yet teams will end up paying, in terms of both players and dollars, as much as four or five times more to get that first player relative to the fourth player. If we look back at the 2004 NFL draft, was Eli Manning really 50 percent better than Philip Rivers and twice as good as Ben Roethlisberger? We could debate the ranking among those three today. Putting aside Roethlisberger’s troubling and well-chronicled “character issues,” most experts and fans probably would rank them in reverse order from their draft selection in terms of value today. You’d be hard put to convince anyone that Manning is appreciably more valuable than Rivers or Roethlisberger; in any event, he’s certainly not twice as valuable. Yet this pattern persists year after year.

  Is having the top pick in the NFL draft such a stroke of good fortune? It’s essentially a coin flip, but not in the traditional sense. Heads, you win a dime; tails, you lose a quarter. Massey and Thaler go so far as to contend that once you factor in salary, the first pick in the entire draft is worth less than the first pick in the second round. (For kicks, imagine the team with the top pick showing up on draft day, the fan base brimming with exuberant optimism, only to hear the commissioner intone: “With the first pick, the Detroit Lions … pass. The Cleveland Browns are now on the clock.”)

  Massey and Thaler discovered another form of overvaluation as well: Teams paid huge prices in terms of future draft picks to move up in the draft. For example, getting a first-round draft pick this year would cost teams two first-round picks the next year or in subsequent years. Gaining an additional second-round pick this year meant giving up the first- and second-round picks next year. Coaches and GMs seemed to put far less value on the future. Looking at all such trades, the so-called implicit discount rate for the future was 174 percent, meaning that teams valued picking today at more than twice and nearly three times the value of taking the same pick in the next year’s draft! Think of this as an interest rate. How many of us would borrow money at an annual interest rate of 174 percent? Even loan sharks aren’t that ruthless.

  Why do NFL teams place so much value on high picks? Psychology explains a lot of it. As anyone who’s ever watched the game show Deal or No Deal or has placed a bid on eBay or at a charity auction can attest, we tend to overpay for an object or a service when we’re in competition with other bidders. We know the value of that $500 gift certificate and raise our paddles accordingly. That’s easy. What if the value of the item is uncertain? When a coveted piece of art or jewelry is up for grabs and the value is unclear, the real bidding war begins, often resulting in overpayment. There’s even a term for this: winner’s curse.

  To demonstrate the winner’s curse, a certain economics professor has been known to stuff a wad of cash into an envelope, stating to his students that there is less than $100 inside. The students bid for its contents; without fail, the winning bid far exceeds the actual contents—sometimes even exceeding $100! (The surplus is used to buy the rest of the students pizza on the last day of class.) The top picks in the NFL draft, by its very nature an exercise in speculation, are singularly ripe for the winner’s curse.

  Here’s another factor in the overpayment of draft picks: As a rule, people are overconfident in their abilities. In all sorts of different contexts, we’re more sure of ourselves than we probably should be. In a well-known study, people were asked at random whether they were above-average drivers. Three-quarters said they were. Similarly, between 75 and 95 percent of money managers, entrepreneurs, and teachers also thought they were “above average” at their job. Not everyone, of course, can be above average. You’d expect roughly half to be. How many times are hiring decisions made on intuition because a boss on the other side of the desk is convinced that in a 30-minute interview she’s found the best candidate? How often do doctors advise a treatment plan because of intuitive decision-making, not because of evidence-based decision-making? They’re just sure they’re right. In the same way, NFL general managers (and sometimes their interfering owners) tend to be overconfident in their ability to assess talent. They trust their gut. Not altogether a bad thing, but they trust it too much and overpay as a result. Never mind the math; they fancy themselves the exception. They know they’re right about this player, just as every entrepreneur knows his business plan is better and every mutual fund manager knows she’s got the winning stock picks.

  Overvaluing because of gut instincts also helps explain why teams invest so much in this year’s pick and so little in next year’s prospects. The guy in front of you is “a once-in-a-lifetime player,” a term invoked almost without fail at every draft. The guy next year is just an abstraction. (Another explanation for the immediacy: GMs and coaches typically have short tenures, so winning now is imperative to keeping their jobs. Even a year can seem beyond their horizon.)

  Overvaluing draft picks isn’t confined to football. In the NBA, teams value this year’s pick at two to three times the value of the same pick in next year’s draft. Collecting data on the NBA draft going back to 1982 and looking at trades for current draft picks that involved future draft picks, we found that future draft picks were discounte
d heavily at 169 percent, almost to the same extent that NFL teams discounted future draft choices. Again, this could be because GMs and coaches have short windows or because teams overvalue what they see now and undervalue what they can’t see readily. Top draft picks in the NBA, however, were only slightly overvalued—not nearly to the extent they are in the NFL. This makes sense: The NBA, after all, has only two draft rounds. Player ability is also easier to predict, there are fewer players and fewer positions to consider, and a single player has a much larger impact on the team than in football. In baseball—where the draft is less important, as so many foreign players sign as free agents—there is also a huge discount applied to future picks, and top picks again tend to be overvalued.

  The truth is that evaluating talent is hard. How hard? For an illustration, consider the case of Eli Manning’s older brother, Peyton. In 1998, Peyton Manning entered the NFL draft with tremendous hype. But teams weren’t sure whether he would be the first or second quarterback taken. There was a comparably touted quarterback from Washington State, Ryan Leaf, considered by many NFL scouts to be the better prospect. Leaf was bigger and stronger than Manning, two easily measurable characteristics, and, again with the support of numbers, was regarded as the better athlete. Although Manning acquitted himself capably at Tennessee, he never led the formidable Volunteers to a national title, a cause for some concern.

  The San Diego Chargers originally held the third pick of the draft but made a trade with the Arizona Cardinals to move up to the second pick to ensure that they got one of the two tantalizing quarterbacks. This move cost them two first-round picks, a second-round pick, reserve linebacker Patrick Sapp, and three-time Pro Bowler Eric Metcalf—all to move up one spot!