Scorecasting: The Hidden Influences Behind How Sports Are Played and Games Are Won
Page 16
If we were consultants to a team equipped with umpire-monitoring technology, our first piece of advice would be: Get rid of QuesTec; it’s wrecking your home field advantage. (How many teams would have agreed so readily to QuesTec if they knew these numbers?) Of course, if we were consulting for MLB, we might have encouraged them to install the technology in all the ballparks or at least tell the umpires that was the case. (Today, that’s essentially what MLB has done.)
We also found the same results for the QuesTec stadiums before and after the system was installed. The called strike-ball differences between home and away teams declined sharply after QuesTec installation, and the decline was particularly pronounced in crucial situations. Even the same umpire behaved differently depending on whether QuesTec was present, calling more strikes and fewer balls on home batters when he was being monitored and doing the opposite when he wasn’t.
Why would the home team advantage for strike-ball calls, particularly in crucial situations, switch completely in the other direction when QuesTec is present? You’d think the advantage would just disappear, creating no bias, but in fact the bias goes in the opposite direction. We suspect that as with the referees in soccer, umpires have to balance the costs and benefits of any bias (conscious or not) they might exhibit. If you know you are being monitored, you want to eliminate any perception of bias. And when the game is on the line, you know that any perceived bias will be scrutinized even more closely. With the speed of the game and the uncertainty of whether a 95-mph fastball hit or missed the outside corner of the plate, umpires may become overly cautious. Worried about accusations of home-team favoritism, the umpire seems to err in the other direction, particularly in situations that will be monitored and analyzed heavily afterward.
What about the other potential umpire biases we found that might benefit the home team, such as stolen bases, double plays, and other close plays? They remain the same whether QuesTec is present or not, which makes sense. After all, QuesTec monitors only the strike zone. It affects no other part of the game. Calls remain in favor of the home team because there’s no “surveillance video” on those calls.
If QuesTec is our smoking gun in the case to prove umpire home team bias in Major League Baseball, Pitch f/x provides the ballistic support.* Using the Pitch f/x location data of the millions of pitches we examined earlier, we asked a series of questions: How likely is it that when the pitch is actually out of the strike zone, an umpire will call a strike on the home team versus the away team? How often is a ball called on a pitch actually within the strike zone when the home team is batting versus the away team, the situation that enraged the die-hard Brewers fan Jack Moore? What about pitches just in or just out of the strike zone? How do these calls change in critical situations?
The following chart graphs the difference in the percentage of called pitches that are called strikes on home versus away batters for pitches within three inches of the dead center of the strike zone, pitches way out of the strike zone (at least three inches), and pitches within 1.5 inches of the strike zone, for example, just on or off the corners. We report the numbers separately for two-strike, three-ball, and full counts.
DIFFERENCE IN PERCENTAGE OF CALLED PITCHES THAT ARE CALLED STRIKES ON HOME VS. AWAY BATTERS
Note two points: (1) The home-away differences are largest for two-strike and three-ball counts and especially for full counts. (2) For the most ambiguous pitches—the ones on the corners—the home-away called-strike discrepancy is largest, which makes sense. The umpire has less discretion over pitches that are less ambiguous. Umpires will be reluctant to make a biased call if the pitch is obviously a strike. In fact, for pitches in the dead center of the strike zone, there is no bias at all. Umpires call these pitches correctly 99 percent of the time whether a home or a visiting batter is standing in front of them. For pitches way outside the strike zone, the umpire has a little more leeway and shows a slight bias in favor of home batters. The umpire has the most discretion for pitches on the corners, and there the home batter bias is largest.
Over the course of a season, all of this adds up to 516 more strikeouts called on away teams and 195 more walks awarded to home teams than there otherwise should be, thanks to the home plate umpire’s bias. And this includes only terminal pitches—where the next called pitch will result in either a strikeout or a walk. Errant calls given earlier in the pitch count could confer an even greater advantage for the home team.
How much do these differences contribute to the home field advantage in baseball? Well, we need to know the value of receiving an extra pitch instead of striking out and the value of being awarded first base instead of facing another pitch, but here’s a rough estimate. Taking the value of a walk and a strikeout in various game situations, this adds up to an extra 7.3 runs per season given to each home team by the plate umpire alone. That might not sound significant but cumulatively, home teams outscore their visitors by only 10.5 runs in a season. Thus, more than two-thirds of the home field advantage in MLB comes by virtue of the home plate umpire’s bad calls.
We can’t expect umpires to be perfect, and in fact, they call strikes and balls correctly 85.6 percent of the time. But the errors they do make don’t seem to be random. They favor the home team.
Now that we understand that there is a bias in called balls and strikes, we get a different understanding of why the home team has better hitting and pitching stats. As we’ve seen, players aren’t hitting or throwing any better at home versus on the road. But when you receive more favorable calls at the plate, this directly improves your hitting numbers. There is also an indirect effect. If home batters are benefiting from more favorable pitch calls, they face more favorable pitch counts and are in a better position to swing at pitches to hit. And when home players are put in these situations, it is more likely that their teammates will be on base when they are at the plate, which gives them more opportunities to produce runs. In short, the direct effect from giving home batters fewer strikes and more balls alone seems to account for a sizable fraction of the home team’s success in MLB. Add to this the indirect benefits and it could well account for just about all of the home team’s advantage.
For evidence of official bias in the NFL, it makes sense to start by considering one obvious component in the control of the men in the striped uniforms: penalties. Home teams receive fewer penalties per game than away teams—about half a penalty less per game—and are charged with fewer yards per penalty. Of course, this does not necessarily mean officials are biased. Away teams might commit more violations and play more sloppily or more aggressively. But when we looked at more crucial situations in the NFL—much as with the Leverage Index or the pitch count in baseball—we found that the penalty bias is exaggerated. It turns out that more valuable penalties, those that result in first downs, also favor the home team.
The most compelling evidence of referee influence in the NFL comes from the introduction of instant replay, which gave coaches—and fans, players, and the media—a chance to review and potentially challenge the call on the field. The inauguration of instant-replay challenge came in 1999, and as with the QuesTec results in baseball, it coincided with a decline in the home team success rate in the NFL, from 58.5 percent (from 1985 to 1998) to 56 percent (from 1999 to 2008), a 29.4 percent drop in the home field advantage. Remember, the home advantage starts only when we get above 50 percent.
Coincidence? We can start by looking at turnovers. First, officials wield considerable influence here because they first determine whether there was a fumble. Second, they determine which team assumes possession of the football. Before instant replay, home teams enjoyed more than an 8 percent edge in turnovers, losing the ball far less often than road teams. When instant replay came along to challenge wrong calls, the turnover advantage was cut in half.
We can also distinguish between fumbles lost (possession changes hands) and fumbles retained (the team with the ball keeps possession). The home team does not actually fumble or drop the ball le
ss often than the away team—in other words, they aren’t “taking care of the ball” any better or worse than the away team. They simply lose fewer fumbles than away teams. After instant replay was installed, however, the home team advantage of losing fewer fumbles miraculously disappeared, whereas the frequency of fumbles remained the same. Home teams are as likely as ever to drop the ball, but now that visiting teams have the ability to challenge the call, home teams aren’t nearly as likely to retain possession.
In close games, when referees’ decisions may really matter—and when the crowd is really involved—home teams enjoyed a healthy 12 percent advantage in recovering fumbles. After instant replay was installed, that advantage simply vanished.
What about penalties? Instant replay is of limited use to us here because teams can’t challenge a penalty call or a noncall. But if we examine the change in penalty discrepancy between home and away teams before and after instant replay, we have a placebo test of sorts. That is, we should not expect to see any changes in penalties. Sure enough, we don’t. The discrepancy in number of penalties and yards per penalty given to home versus away teams hardly changed after instant replay. This helps confirm that it is instant replay, not something else, that has driven the recent changes in turnovers and winning percentage of home teams in the NFL.
If referee bias is driving these patterns and instant replay mitigated these biases, we should see that visiting teams are more successful when they challenge a referee’s call using instant replay. In other words, if away teams are indeed getting more bad calls than home teams, more of those calls will be overturned on instant replay. We looked at the results of nearly 1,300 instant-replay challenges from 2005 to 2009 to examine the success rate of home team challenges versus away team challenges.
The results? It turns out that away teams are indeed more successful in overturning a call than home teams are, but only modestly so (37 percent versus 35 percent). Both are slightly more successful than official challenges (33 percent), which are challenges initiated by an official in the last two minutes of each half on close plays. These statistics are misleading, though, because as we saw in baseball and soccer, referees are less likely to make biased judgments when the game is no longer in doubt. So what happens if the home team is behind? When the home team is losing, a challenge made by the home team is successful 28.4 percent of the time. But a challenge made by the away team is successful 40.0 percent of the time. Thus, away teams seem to be getting more than their fair share of bad calls when they are winning, which is when bad calls would be most valuable to the home team.
Could referee bias explain a large part of the home field advantage in football? Absolutely. Again we see a dramatic reduction in the home team’s edge when instant replay is introduced. Yet instant replay affords each team only a maximum of three incorrect challenges per game and is limited to certain circumstances. Clearly there are other calls not eligible for challenge that could favor the home team, such as penalties. The fact that home teams in football have better offensive stats—such as rushing more successfully and having longer time of possession—could be the result of getting more favorable calls, fewer penalties, and fewer turnovers. If you play at home and sense that you’re less likely to get called for a penalty, you may be more inclined to block much more aggressively or challenge a receiver.
Recall that in the NBA home and away teams shoot identically from the free throw line. But home teams shoot more free throws than away teams—between 1 and 1.5 more per game. Why? Because away teams are called for more fouls, particularly shooting fouls. Away teams also are called for more turnovers and more violations. These differences could be caused by more aggressive or sloppy play on the part of road teams, which could be more tired because of the lopsided NBA schedule. But they are also consistent with referee bias.
To help distinguish sloppy play on the road from referee home bias, let’s take a closer look at the types of fouls, turnovers, and violations that are committed by home and away teams. Certain fouls, turnovers, and violations require more referee discretion and judgment than others. For example, highly uncertain situations and close calls, where a judgment must be made, allow for greater referee influence, as opposed to something less ambiguous such as a shot clock violation that everyone can easily monitor because the 24-second shot clock is posted above the two baskets and a red light illuminates the glass backboard when the clock expires.
If sloppy or aggressive play by the away team is causing these differences, we should not expect to see the number of violations vary with how ambiguous or uncertain the fouls, turnovers, or violations are regardless of how much referee judgment is required. If you’re playing badly, you’re probably playing badly across many dimensions of the game.
We looked at calls requiring more or less referee judgment to see whether the home advantage was the same. Loose ball and offensive fouls seem to be the most ambiguous and contentious. Ted Bernhardt, a longtime NBA official, now retired, helped us with our analysis. “Blocking fouls versus charging fouls are by far the hardest calls to make,” he says. It turns out that offensive and loose ball fouls go the home team’s way at twice the rate of other personal fouls. We can also look at fouls that are more valuable, such as those that cause a change of possession. These fouls are almost four times more likely to go the home team’s way than fouls that don’t cause a change of possession.
What about turnovers and violations? Turnovers from shot clock violations, which aren’t particularly ambiguous or controversial, are no different for home or away teams. Turnovers from five-second violations on inbounds plays, which are also fairly unambiguous because everyone can count (though referees may count a little slower or faster than everyone else and there is no clock indicating when five seconds has elapsed), are also not very different for home and away teams (in fact, home teams receive slightly more five-second violations).
If, however, we look at the most ambiguous turnover calls requiring the most judgment, such as palming and traveling, we see huge differences in home and away numbers. The chance of a visiting player getting called for traveling is 15 percent higher than it is for a home team player. The fact that ambiguous fouls and turnovers tend to go the home team’s way and unambiguous ones don’t is hard to reconcile with sloppy play on the part of visiting teams. But it’s exactly what you would expect from referee bias.
Identifying refereeing bias in the NBA is especially hard because context is so important, and some of the most controversial “calls” in basketball are in fact “no calls”—when a call is not made. But the evidence seems to suggest ref bias toward the home team. If bias clearly exists in soccer, baseball, and football, isn’t it reasonable to suspect that NBA referees are vulnerable to the same influences?
Remember the Portland Trail Blazers playing so sluggishly in that dreary midweek road loss to the San Antonio Spurs? On the road in San Antonio, the Trail Blazers committed 13 fouls, the Spurs 14; each team had six turnovers. But if we look at the types of fouls and turnovers over which referees have more influence, we see that the Blazers were whistled for twice as many loose ball fouls as the Spurs and that five of the Blazers’ six turnovers were on judgment calls made by the referee (one traveling, two ambiguous lost balls out of bounds, one offensive goaltending, and one questionable kicked ball). By contrast, all six Spurs turnovers were unambiguous (five bad passes/steals and one shot clock violation). In addition, more of the calls against the Blazers resulted in a change of possession favoring the Spurs. Perhaps it’s not so surprising that the Spurs won.
Recall that only a few nights later the teams met again, this time in Portland. The Blazers won by 18 points. The same advantages conferred on the home team were present, though this time it was the Trail Blazers who were the beneficiaries. The Spurs were whistled for 25 fouls and 16 turnovers, compared with Portland, which had 18 fouls and 13 turnovers. The types of foul calls and turnovers tell an even stronger story. Among the visiting Spurs’ 16 turnovers, 11 were of th
e more ambiguous variety, including a couple of debatable lost balls out of bounds, and two Spurs players were even called for palming. (To give you an idea of how rarely palming is called in the NBA, on average there is one palming call every five or six games.) Of the Blazers’ 13 turnovers, 10 were unambiguous, consisting of two shot clock violations and bad passes that were stolen or thrown out of bounds. There were ten situations in which the ball was tipped out of bounds—eight went the Trail Blazers’ way. More fouls resulting in a change of possession went Portland’s way as well. If we tally the numbers across the two games, ambiguous turnovers went the home team’s way 85 percent of the time, ambiguous fouls were charged to the visiting team 72 percent of the time, and the home team won by a collective 33-point margin.
So how much of the home court advantage in the NBA is due to referee bias? If we attribute the differences in free throw attempts to referee bias, this would account for 0.8 points per game. That alone accounts for almost one-fourth of the NBA home court advantage of 3.4 points per game. If we gave credit to the referees for the more ambiguous turnover differences and computed the value of those turnovers, this would also capture another quarter of the home team’s advantage. Attributing some of the other foul differences to the referees and adding the effects of those fouls (other than free throws) on the game, this brings the total to about three-quarters of the home team’s advantage. And remember, scheduling in the NBA explained about 21 percent of the home team’s success, as well. That adds up to nearly all of the NBA home court advantage.
Long story short, referee bias could well be the main reason for home court advantage in basketball. And if the refs call turnovers and fouls in the home team’s favor, we can assume they make other biased calls in favor of the home team that we cannot see or measure.