Full House +xtras

by Stephen Jay Gould

  10

  Why the Death of 0.400 Hitting Records Improvement of Play

  So far I have only demonstrated a pattern based on unconventional concepts and pictures. I have not yet proposed an explanation. I have proposed that 0.400 hitting be reconceptualizcd as an inextricable segment in a full house of variation—as the right tail of the bell curve of batting averages—and not as a self-contained entity whose disappearance must record the degeneration of batting in some form or other.

  In this different model and picture, 0.400 hitting disappears as a consequence of shrinking variation around a stable mean batting average. The shrinkage is so exceptionless, so apparently lawlike in its regularity, that we must be discerning something general about the behavior of systems through time.

  Why should such a shrinkage of variation record the worsening of anything? The final and explanatory step in my argument must proceed beyond the statistical analysis of batting averages. We must consider both the nature of" baseball as a system, and some general properties of systems that enjoy long persistence with no major changes in procedures and behaviors. I therefore devote this section to reasons for celebrating the loss of 0.400 hitting as a mark of better baseball.

  Two arguments, and supporting data, convince me that shrinkage of variation (with consequent disappearance of 0.400 hitting) must be measuring a general improvement of play. The two formulations sound quite dissimilar at first, but really represent different facets of a single argument.

  1. _Complex systems improve when the best performers play by the same rules over extended periods of time. As systems improve, they equilibrate and variation decreases_. No other major American sport permits such an analysis, for all others have changed their fundamental rules too often and too recently. As a teenager, I played basketball without the twenty-four-second rule. My father played with a center jump after each basket. His father (had he been either inclined or acculturated) would have brought the ball down-court with a two-handed dribble. And Mr. Naismith's boys threw the ball into a peach basket. While the peach basket still hung in the 1890s, baseball made its last major change in procedure (as discussed in the last chapter) by moving the pitcher's mound back to the current distance of sixty feet six inches.

  But constant rules don't imply unchanging practices. (In the last chapter I discussed the numerous fiddlings and jigglings imposed by rule-makers to keep pitching and hitting in balance.) Dedicated performers are constantly watching, thinking, and struggling for ways to twiddle or manipulate the system in order to gain a legitimate edge (new techniques for hitting a curve, for gobbling up a ground ball, for gyrating in a windup to fool the batter). Word spreads, and these minor discoveries begin to pervade the system. The net result through time must inevitably encourage an ever-closer approach to optimal performance in all aspects of play—combined with ever-decreasing variation in modes of procedure.

  Baseball was feeling its juvenile way during the early days of major league play. The basic rules of the 1890s are still our rules, but scores of subtleties hadn't yet been invented or developed. Rough edges careered out in all directions from a stable center. To cite just a few examples (taken from Bill James's _Historical Baseball Abstract_): pitchers only began to cover first base in the 1890s. During the same decade, Brooklyn developed the cutoff play, while the Boston Beaneaters invented the hit-and-run, and signals from runner to batter. Gloves were a joke in these early days—just a bit of leather over the hand, not today's baskets for trapping balls. As a fine symbol of broader tolerance and variation, the 1896 Philadelphia Phillies actually experimented for seventy-three games with a lefty shortstop. Unsurprisingly, traditional wisdom applied. He stank—turning in the worst fielding average with the fewest assists among all regular shortstops in the league.

  In baseball's youth, styles of play had not become sufficiently regular and optimized to foil the accomplishments of the very best. Wee Willie Keeler could "hit 'em where they ain't" (his motto), and compile a 0.432 batting average in 1897, because fielders didn't yet know where they should be. Slowly, by long distillation of experience, players moved toward optimal methods of positioning, fielding, pitching, and batting—and variation inevitably declined. The best now meet an opposition too finely honed to its own perfection to permit the extremes of accomplishment that characterized a more casual and experimental age. We cannot explain the disappearance of 0.400 hitting simply by saying (however true) that managers invented relief pitching, while pitchers invented the slider—for such traditional explanations abstract 0.400 hitting as an independent phenomenon and view its extinction as the chief sign of a trend to deterioration in bailing. Rather, hitting has improved along with all other aspects of play as the entire game sharpened its standards, narrowed its ranges of tolerance, and therefore limited variation in performance as all parts of the game climbed a broader-based hill toward a much narrower pinnacle.

  Consider the predicament of a modern Wade Boggs, Tony Gwynn, Rod Carew, or George Brett. Can anyone truly believe that these great hitters are worse than Wee Willie Keeler (at five feet four and a half inches and 140 pounds), Ty Cobb, or Rogers Hornsby? Every pitch is now charted, every hit mapped to the nearest square inch. Fielding and relaying have improved dramatically. Fresh and rested pitching arms must be faced in the late innings; fielders scoop up grounders in gloves as big as a brontosaurus's footprint. Relative to the right wall of human limitation. Tony Gwynn and Wee Willie Keeler must stand in the same place—just a few inches from theoretical perfection (the best that human muscles and bones can do). But average play has so crept up upon Gwynn that he lacks the space for taking advantage of suboptimality in others. All these general improvements must rob great batters of ten to twenty hits a year—a bonus that would be more than enough to convert any of the great modern hatters info 0.400 hitters.

  I have formulated the argument parochially in the terms and personnel of baseball. But I feel confident that I am describing a general property of systems composed of individual units competing with one another _ under stable rules and for prizes of victory. Individual players struggle to rind means for improvement—up to limits imposed by balances of competition and mechanical properties of materials—and their discoveries accumulate within the system, leading to general gains toward an optimum. As the system nears this narrow pinnacle, variation must decrease—for only the very best can now enter, while their predecessors have slowly, by trial and error, discovered better procedures that now cannot be substantially improved. When someone discovers a truly superior way, everyone else copies and variation diminishes.

  Thus I suspect that similar reasons (along with a good dollop of historical happenstance) govern the uniformity of automotive settling upon internal combustion engines from a wider set of initial possibilities including steam and electric power; the Standardization of business practices; the reduction of life's initial multicellular animal diversity to just a handful of major phyla (see Gould, 1989); and the disappearance of 0.400 hitting in baseball as variation shrinks symmetrically around a stable mean batting average.

  In the good old days of greater variation and poorer play, you could get a job for "good field, no hit”—but no longer as the game improved and the pool of applicants widened. So the left tail shriveled up and moved toward the mean. In those same legendary days, the very best hitters could take advantage of a sloppier system that had not yet discovered optlmalities of opposing activities in fielding and pitching. Our modern best hitters are just as good and probably better, but average pitching and fielding have so improved that the truly superb cannot soar so far above the ordinary. Therefore the right tail shriveled up and also moved toward the mean.

  I first published these ideas in the initial issue of the revived _Vanity Fair_ in March 1983. To my gratification, several fellow sabermetricians became intrigued and took up the challenge to test my ideas with other sources of baseball data. The results have been most gratifying. In particular, my colleagues have provided good examples of
the two most important predictions made by models for general improvement marked by decreasing variation.

  _Specialization and division of labor_. Ever since Adam Smith began _The Wealth of Nations_ with his famous example of pinmaking, specialization and division of labor have been viewed as the major criteria of increasing efficiency and approach to optimality. In their paper "On the tendency toward increasing specialization following the inception of a complex system—professional baseball 1871-1988," John Fellows, Pete Palmer, and Steve Mann plotted the number of major leaguers who played more than one fielding position in a single season. Note (see Figure 17) the steady decrease and subsequent stabilization, a pattern much like the decelerating decline of standard deviations in Figure 16—though in this case measuring the increase of specialization through baseball's history (I do not know why values rose slightly in the 1960s, though to nowhere near the high levels of baseball's early history).

  _Decreasing variation_. My colleagues Sangit Chatterjee and Mustafa Yilinaz of the College of Business Administration at Northeastern University (baseball does provide some wonderful cohesion amid our diversity) wrote an article on "Parity in baseball: stability of evolving systems." In searching for an example even more general than shrinking variation in batting averages, Chatterjee and Yilmaz reasoned that if general play has improved, with less variation among a group of consistently better players, then disparity among teams should also decrease—that is, the difference between the best and worst clubs should decline because all reams can now fill their rosters with enough good players, leading to greater equalization through time. The authors therefore plotted the standard deviation in seasonal winning percentage from the beginning of major league baseball to the present. Figure 18 shows a steady fall in standard deviation, indicating a decreasing difference between the best and worst teams through the history of play.[7]

  [7. These statistics can also he broken down to yield finer patterns that validate the hypothesis. The National League began in 1876. the American in 1901. Since the hypothesis holds that systems equilibrate through time by decelerating decrease in variation, we might predict that, from 1901 to 1930, when the American League was new but the National already in middle age, variation in American League records should decrease more rapidly than comparable measures in the National League. This pattern does indeed emerge, both for standard deviations of batting averages in my calculations. And for the history of differences in best versus worst teams in the data of Chatterjee arid Yilmaz.]

  2. _As play improves and bell curves march toward the right wall, variation must shrink at the right tail_. I discussed the notion of "walls" in chapter 4—upper and lower limits to variation imposed by laws of nature, structure of materials, etc. (There I illustrated a minimal left wall in the story of my medical history—an obvious and logical lower bound of zero time between diagnosis and death from the same disease. Part Four will focus upon a left wall of minimal complexity for life—for nothing much simpler than a bacterial cell could be preserved in the fossil record.) We would all, I think, accept the notion that a "right wall" must exist for human achievement. We cannot, after all, perform beyond the limits of what human hone and muscle can accomplish; no man will ever outpace a cheetah or a finch. We would also, I assume, acknowledge that some extraordinary people, by combination of generic gift, maniacal dedication, and rigorous training, push their bodies to perform as close to the right wall as human achievement will allow.

  Earlier I discussed the major phenomenon in sports that must be signaling approach to the right wall—a flattening out of improvement (measured by record breaking) as sports mature, promise ever greater rewards, become accessible to all, and optimize methods of training (see pp. 92—97). This flattening out must represent the approach of the best to the right wall. The longer a sport has endured with stable rules and maximal access, the closer the best should stand to the right wall, and the less we should therefore expect any sudden and massive breaking of records. When George Plimpton, several years ago, wrote about a great pitching prospect who could throw 140 miles per hour, all serious fans recognized this essay in "straight" reporting as a spoof, though many less knowledgeable folks were fooled. From Walter Johnson in the 1920s to Nolan Ryan today, the best fastball pitchers have tried to throw at maximal speed, and no one has consistently broken 100 mph. In fact, Johnson was probably as fast as Ryan. Thus, we can assume that these men stand near the right wall of what a human arm can do. Barring some unexpected invention in technique, no one is going to descend from some baseball Valhalla and start throwing 40 percent again as fast—not after a century of trying among the very best.

  These approaches to the right wall can easily be discerned in sports that keep absolute records measured as times and distances. As previously discussed, record times for the marathon, or virtually any other timed event with stable rules and no major innovations, drop steadily—in the decelerating pattern of initial rapidity, followed by later plateauing as the best draw near to the right wall. But this pattern is masked in baseball, because most records measure one activity relative to another, and not against an absolute standard of rime or distance. Batting records mark what a hitter does against pitchers. A mean league batting average of 0.260 is not an absolute measure of anything, but a general rate of success for hitters versus pitchers. Therefore a fall or rise in mean batting average does not imply that hitters are becoming absolutely worse or better, but only that their performance relative to pitchers has changed.

  Thus we have been fooled in reading baseball records. We note that the mean batting average has never strayed much from 0.260, and we therefore wrongly assume that batting skills have remained in a century-long rut. We note that 0.400 hitting has disappeared, and we falsely assume that great hitting has gone belly-up. But when we recognize these averages as relative records, and acknowledge that baseball professionals, like all other premier athletes, must be improving with time, a different (and almost surely correct) picture emerges (see Figure 19)—one that acknowledges batting averages as components in a full house of variation with a bell-curve distribution and that, as an incidental consequence or no mean importance, allows us finally to visualize why the extinction of 0.400 hitting must be measuring improvement of play as marked by shrinking variation.

  Early in the history of baseball (top part of Figure 19), average play stood far from the right wall of human limits. Both hitters and pitchers performed considerably below modern standards, but the balance between them did not differ from today's—and we measure this unchanging balance as 0.260 hitting. Thus, in these early days, the mean batting average of 0.260 fell well below the right wall, and variation spread out widely on both sides—at the lower end, because the looser and less accomplished system did provide jobs to good fielders who couldn't hit; and at the upper end, because so much space existed between the average and the right wall.

  A few men of extraordinary talent and dedication always push their skills to the very limit of human accomplishment and reside near the right wall. In baseball's early days, these men stood so far above the mean that we measured their superior performance as 0.400 batting.

  Consider what has happened to modern baseball (lower part of Figure 19). General play has improved significantly in all aspects of the game. But the balance between hitting and pitching has not altered. (I showed on pp. 101—105 that the standard-bearer’s of baseball have frequently fiddled with the rules in order to maintain this balance.) The mean batting average has therefore remained constant, but this stable number represents markedly superior performance today (in both hitting and pitching). Therefore, this unchanged average must now reside much closer to the right wall. Meanwhile, and inevitably, variation in the entire system has shriveled symmetrically on both sides—at the lower end, because improvement of play now debars employment to men who field well but cannot hit; and at the upper end. for the simple reason that much less room now exists between the upwardly mobile mean and the unchanging right wall. The t
op hitters, trapped at the upper bound of the right wall, must now lie closer to the mean than did their counterparts of yore.

  The best hitters of today can't be worse than 0.400 hitters of the past. In fact, the modern stars may have improved slightly and may now stand an inch or two closer to the right wall. But the average player has moved several feet closer to the right wall—and the distance between ordinary (maintained at 0.260) and best has decreased, thereby erasing batting averages as high as 0.400. Ironically, therefore, the disappearance of 0.400 hitting marks the general improvement of play, not a decline In anything.

  Our confidence in this explanation will increase if supporting data can be provided with statistics for other aspects of play through time. I have compiled similar records for the other two major facets of baseball—fielding and pitching. Both support the key predictions of a model that posits increasing excellence of play with decreasing variation when the best can no longer take such numerical advantage of the poorer quality in average performance.

  Most batting and pitching records are relative, but the primary measure of good fielding is absolute (or at least effectively so). A fielding average is you against the ball, and I don't think that grounders or fly balls have improved through time (though the hitters have). I suspect that modem fielders are trying to accomplish the same tasks, at about the same level of difficulty, as their older counterparts. Fielding averages (the percent of errorless chances) should therefore provide an absolute measure of changing excellence in play. If baseball has improved, we should note a decelerating rise in fielding averages through time. (I do recognize that some improvement might be attributed to changing conditions, rather than absolutely improving play, just as some running records may fall because modern tracks are better raked and pitched. Older infields were, apparently, lumpier and bumpier than the productions of good ground crews today—so some of the poorer fielding of early days may have resulted from lousy fields rather than lousy fielders. I also recognize that rising averages must be tied in large part to great improvement in the design of gloves—but better equipment represents a major theme of history, and one of the legitimate reasons underlying my claim for general improvement in play.)