Bully for Brontosaurus

by Stephen Jay Gould

  We understand Hutton’s wisdom. We do not search for unattainable ultimates. We define evolution, using Darwin’s phrase, as “descent with modification” from prior living things. Our documentation of life’s evolutionary tree records one of science’s greatest triumphs, a profoundly liberating discovery on the oldest maxim that truth can make us free. We have made this discovery by recognizing what can be answered and what must be left alone. If Justice Scalia heeded our definitions and our practices, he would understand why creationism cannot qualify as science. He would also, by the way, sense the excitement of evolution and its evidence; no person of substance could be unmoved by something so interesting. Only Aristotle’s creator may be so impassive.

  Don Quixote recognized “no limits but the sky,” but became thereby the literary embodiment of unattainable reverie. G. K. Chesterton understood that any discipline must define its borders of fruitfulness. He spoke for painting, but you may substitute any creative enterprise: “Art is limitation: the essence of every picture is the frame.”

  9 | Numbers and Probability

  31 | The Streak of Streaks*

  MY FATHER was a court stenographer. At his less than princely salary, we watched Yankee games from the bleachers or high in the third deck. But one of the judges had season tickets, so we occasionally sat in the lower boxes when hizzoner couldn’t attend. One afternoon, while DiMaggio was going 0 for 4 against, of all people, the lowly St. Louis Browns, the great man fouled one in our direction. “Catch it, Dad,” I screamed. “You never get them,” he replied, but stuck up his hand like the Statue of Liberty—and the ball fell right in. I mailed it to DiMaggio, and, bless him, he actually sent the ball back, signed and in a box marked “insured.” Insured, that is, to make me the envy of the neighborhood, and DiMaggio the model and hero of my life.

  I met DiMaggio a few years ago on a small playing field at the Presidio of San Francisco. My son, wearing DiMaggio’s old number 5 on his Little League jersey, accompanied me, exactly one generation after my father caught that ball. DiMaggio gave him a pointer or two on batting and then signed a baseball for him. One generation passeth away, and another generation cometh: But the earth abideth forever.

  My son, uncoached by Dad, and given the chance that comes but once in a lifetime, asked DiMaggio as his only query about life and career: “Suppose you had walked every time up during one game of your 56-game hitting streak? Would the streak have been over?” DiMaggio replied that, under 1941 rules, the streak would have ended, but that this unfair statute has since been revised, and such a game would not count today.

  My son’s choice for a single question tells us something vital about the nature of legend. A man may labor for a professional lifetime, especially in sport or in battle, but posterity needs a single transcendant event to fix him in permanent memory. Every hero must be a Wellington on the right side of his personal Waterloo; generality of excellence is too diffuse. The unambiguous factuality of a single achievement is adamantine. Detractors can argue forever about the general tenor of your life and works, but they can never erase a great event.

  In 1941, as I gestated in my mother’s womb, Joe DiMaggio got at least one hit in each of 56 successive games. Most records are only incrementally superior to runners-up; Roger Maris hit 61 homers in 1961, but Babe Ruth hit 60 in 1927 and 59 in 1921, while Hank Greenberg (1938) and Jimmy Foxx (1932) both hit 58. But DiMaggio’s 56-game hitting streak is ridiculously, almost unreachably far from all challengers (Wee Willie Keeler and Pete Rose, both with 44, come second). Among sabermetricians (a happy neologism based on an acronym for members of the Society for American Baseball Research, and referring to the statistical mavens of the sport)—a contentious lot not known for agreement about anything—we find virtual consensus that DiMaggio’s 56-game hitting streak is the greatest accomplishment in the history of baseball, if not all modern sport.

  The reasons for this respect are not far to seek. Single moments of unexpected supremacy—Johnny Vander Meer’s back-to-back no-hitters in 1938, Don Larsen’s perfect game in the 1956 World Series—can occur at any time to almost anybody, and have an irreducibly capricious character. Achievements of a full season—such as Maris’s 61 homers in 1961 and Ted Williams’s batting average of .406, also posted in 1941 and not equaled since—have a certain overall majesty, but they don’t demand unfailing consistency every single day; you can slump for a while, so long as your average holds. But a streak must be absolutely exceptionless; you are not allowed a single day of subpar play, or even bad luck. You bat only four or five times in an average game. Sometimes two or three of these efforts yield walks, and you get only one or two shots at a hit. Moreover, as tension mounts and notice increases, your life becomes unbearable. Reporters dog your every step; fans are even more intrusive than usual (one stole DiMaggio’s favorite bat right in the middle of his streak). You cannot make a single mistake.

  Thus Joe DiMaggio’s 56-game hitting streak is both the greatest factual achievement in the history of baseball and a principal icon of American mythology. What shall we do with such a central item of our cultural history?

  Statistics and mythology may strike us as the most unlikely of bedfellows. How can we quantify Caruso or measure Middlemarch? But if God could mete out heaven with the span (Isaiah 40:12), perhaps we can say something useful about hitting streaks. The statistics of “runs,” defined as continuous series of good or bad results (including baseball’s streaks and slumps), is a well-developed branch of the profession, and can yield clear—but wildly counterintuitive—results. (The fact that we find these conclusions so surprising is the key to appreciating DiMaggio’s achievement, the point of this article, and the gateway to an important insight about the human mind.)

  Start with a phenomenon that nearly everyone both accepts and considers well understood—“hot hands” in basketball. Now and then, someone just gets hot, and can’t be stopped. Basket after basket falls in—or out as with “cold hands,” when a man can’t buy a bucket for love or money (choose your cliché). The reason for this phenomenon is clear enough: It lies embodied in the maxim, “When you’re hot, you’re hot; and when you’re not, you’re not.” You get that touch, build confidence; all nervousness fades, you find your rhythm; swish, swish, swish. Or you miss a few, get rattled, endure the booing, experience despair; hands start shaking and you realize that you shoulda stood in bed.

  Everybody knows about hot hands. The only problem is that no such phenomenon exists. Stanford psychologist Amos Tversky studied every basket made by the Philadelphia 76ers for more than a season. He found, first of all, that the probability of making a second basket did not rise following a successful shot. Moreover, the number of “runs,” or baskets in succession, was no greater than what a standard random, or coin-tossing, model would predict. (If the chance of making each basket is 0.5, for example, a reasonable value for good shooters, five hits in a row will occur, on average, once in 32 sequences—just as you can expect to toss five successive heads about once in 32 times, or 0.55.)

  Of course Larry Bird, the great forward of the Boston Celtics, will have more sequences of five than Joe Airball—but not because he has greater will or gets in that magic rhythm more often. Larry has longer runs because his average success rate is so much higher, and random models predict more frequent and longer sequences. If Larry shoots field goals at 0.6 probability of success, he will get five in a row about once every 13 sequences (0.65). If Joe, by contrast, shoots only 0.3, he will get his five straight only about once in 412 times. In other words, we need no special explanation for the apparent pattern of long runs. There is no ineffable “causality of circumstance” (to coin a phrase), no definite reason born of the particulars that make for heroic myths—courage in the clinch, strength in adversity, etc. You only have to know a person’s ordinary play in order to predict his sequences. (I rather suspect that we are convinced of the contrary not only because we need myths so badly, but also because we remember the successes and simply allow the
failures to fade from memory. More on this later.) But how does this revisionist pessimism work for baseball?

  My colleague Ed Purcell, Nobel laureate in physics but, for purposes of this subject, just another baseball fan, has done a comprehensive study of all baseball streak and slump records. His firm conclusion is easily and swiftly summarized. Nothing ever happened in baseball above and beyond the frequency predicted by coin-tossing models. The longest runs of wins or losses are as long as they should be, and occur about as often as they ought to. Even the hapless Orioles, at 0 and 21 to start the 1988 season, only fell victim to the laws of probability (and not to the vengeful God of racism, out to punish major league baseball’s only black manager).*

  But “treasure your exceptions,” as the old motto goes. Purcell’s rule has but one major exception, one sequence so many standard deviations above the expected distribution that it should never have occurred at all: Joe DiMaggio’s 56-game hitting streak in 1941. The intuition of baseball aficionados has been vindicated. Purcell calculated that to make it likely (probability greater than 50 percent) that a run of even 50 games will occur once in the history of baseball up to now (and 56 is a lot more than 50 in this kind of league), baseball’s rosters would have to include either four lifetime .400 batters or 52 lifetime .350 batters over careers of 1,000 games. In actuality, only three men have lifetime batting averages in excess of .350, and no one is anywhere near .400 (Ty Cobb at .367, Rogers Hornsby at .358, and Shoeless Joe Jackson at .356). DiMaggio’s streak is the most extraordinary thing that ever happened in American sports. He sits on the shoulders of two bearers—mythology and science. For Joe DiMaggio accomplished what no other ballplayer has done. He beat the hardest taskmaster of all, a woman who makes Nolan Ryan’s fastball look like a cantaloupe in slow motion—Lady Luck.

  A larger issue lies behind basic documentation and simple appreciation. For we don’t understand the truly special character of DiMaggio’s record because we are so poorly equipped, whether by habits of culture or by our modes of cognition, to grasp the workings of random processes and patterning in nature.

  Omar Khayyám, the old Persian tentmaker, understood the quandary of our lives (Rubaiyat of Omar Khayyám, Edward Fitzgerald, trans.):

  Into this Universe, and Why not knowing,

  Nor Whence, like Water willy-nilly flowing;

  And out of it, as Wind along the Waste,

  I know not Whither, willy-nilly blowing.

  But we cannot bear it. We must have comforting answers. We see pattern, for pattern surely exists, even in a purely random world. (Only a highly nonrandom universe could possibly cancel out the clumping that we perceive as pattern. We think we see constellations because stars are dispersed at random in the heavens, and therefore clump in our sight—see Essay 17.) Our error lies not in the perception of pattern but in automatically imbuing pattern with meaning, especially with meaning that can bring us comfort, or dispel confusion. Again, Omar took the more honest approach:

  Ah, love! could you and I with Fate conspire

  To grasp this sorry Scheme of Things entire,

  Would not we shatter it to bits—and then

  Re-mould it nearer to the Heart’s Desire!

  We, instead, have tried to impose that “heart’s desire” upon the actual earth and its largely random patterns (Alexander Pope, Essay on Man, end of Epistle 1):

  All Nature is but Art, unknown to thee;

  All Chance, Direction, which thou canst not see;

  All Discord, Harmony not understood:

  All partial Evil, universal Good.

  Sorry to wax so poetic and tendentious about something that leads back to DiMaggio’s hitting streak, but this broader setting forms the source of our misinterpretation. We believe in “hot hands” because we must impart meaning to a pattern—and we like meanings that tell stories about heroism, valor, and excellence. We believe that long streaks and slumps must have direct causes internal to the sequence itself, and we have no feel for the frequency and length of sequences in random data. Thus, while we understand that DiMaggio’s hitting streak was the longest ever, we don’t appreciate its truly special character because we view all the others as equally patterned by cause, only a little shorter. We distinguish DiMaggio’s feat merely by quantity along a continuum of courage; we should, instead, view his 56-game hitting streak as a unique assault upon the otherwise unblemished record of Dame Probability.

  Amos Tversky, who studied “hot hands,” has performed, with Daniel Kahneman, a series of elegant psychological experiments. These long-term studies have provided our finest insight into “natural reasoning” and its curious departure from logical truth. To cite an example, they construct a fictional description of a young woman: “Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.” Subjects are then given a list of hypothetical statements about Linda: They must rank these in order of presumed likelihood, most to least probable. Tversky and Kahneman list eight statements, but five are a blind, and only three make up the true experiment:

  Linda is active in the feminist movement;

  Linda is a bank teller;

  Linda is a bank teller and is active in the feminist movement.

  Now it simply must be true that the third statement is least likely, since any conjunction has to be less probable than either of its parts considered separately. Everybody can understand this when the principle is explained explicitly and patiently. But all groups of subjects, sophisticated students who have pondered logic and probability as well as folks off the street corner, rank the last statement as more probable than the second. (I am particularly fond of this example because I know that the third statement is least probable, yet a little homunculus in my head continues to jump up and down, shouting at me—“but she can’t just be a bank teller; read the description.”)

  Why do we so consistently make this simple logical error? Tversky and Kahneman argue, correctly I think, that our minds are not built (for whatever reason) to work by the rules of probability, though these rules clearly govern our universe. We do something else that usually serves us well, but fails in crucial instances: We “match to type.” We abstract what we consider the “essence” of an entity, and then arrange our judgments by their degree of similarity to this assumed type. Since we are given a “type” for Linda that implies feminism, but definitely not a bank job, we rank any statement matching the type as more probable than another that only contains material contrary to the type. This propensity may help us to understand an entire range of human preferences, from Plato’s theory of form to modern stereotyping of race or gender.

  We might also understand the world better, and free ourselves of unseemly prejudice, if we properly grasped the workings of probability and its inexorable hold, through laws of logic, upon much of nature’s pattern. “Matching to type” is one common error; failure to understand random patterning in streaks and slumps is another—hence Tversky’s study of both the fictional Linda and the 76ers’ baskets. Our failure to appreciate the uniqueness of DiMaggio’s streak derives from the same unnatural and uncomfortable relationship that we maintain with probability. (If we knew Lady Luck better, Las Vegas might still be a roadstop in the desert.)

  My favorite illustration of this basic misunderstanding, as applied to DiMaggio’s hitting streak, appeared in a recent article by baseball writer John Holway, “A Little Help from His Friends,” and subtitled “Hits or Hype in ’41” (Sports Heritage, 1987). Holway points out that five of DiMaggio’s successes were narrow escapes and lucky breaks. He received two benefits-of-the-doubt from official scorers on plays that might have been judged as errors. In each of two games, his only hit was a cheapie. In game 16, a ball dropped untouched in the outfield and had to be called a hit, even though the ball had been misjudged and could have been caught; in game 54, DiMaggio dribbled one down the
third-base line, easily beating the throw because the third baseman, expecting the usual, was playing far back. The fifth incident is an oft-told tale, perhaps the most interesting story of the streak. In game 38, DiMaggio was 0 for 3 going into the last inning. Scheduled to bat fourth, he might have been denied a chance to hit at all. Johnny Sturm popped up to begin the inning, but Red Rolfe then walked. Slugger Tommy Henrich, up next, was suddenly swept with a premonitory fear: Suppose I ground into a double play and end the inning? An elegant solution immediately occurred to him: Why not bunt (an odd strategy for a power hitter). Henrich laid down a beauty; DiMaggio, up next, promptly drilled a double to left.

  I enjoyed Holway’s account, but his premise is entirely, almost preciously, wrong. First of all, none of the five incidents represents an egregious miscall. The two hits were less than elegant, but undoubtedly legitimate; the two boosts from official scorers were close calls on judgment plays, not gifts. As for Henrich, I can only repeat manager Joe McCarthy’s comment when Tommy asked him for permission to bunt: “Yeah, that’s a good idea.” Not a terrible strategy either—to put a man into scoring position for an insurance run when you’re up 3–1.

  But these details do not touch the main point: Holway’s premise is false because he accepts the conventional mythology about long sequences. He believes that streaks are unbroken runs of causal courage—so that any prolongation by hook-or-crook becomes an outrage against the deep meaning of the phenomenon. But extended sequences are not pure exercises in valor. Long streaks always are, and must be, a matter of extraordinary luck imposed upon great skill. Please don’t make the vulgar mistake of thinking that Purcell or Tversky or I or anyone else would attribute a long streak to “just luck”—as though everyone’s chances are exactly the same, and streaks represent nothing more than the lucky atom that kept moving in one direction. Long hitting streaks happen to the greatest players—Sisler, Keeler, DiMaggio, Rose—because their general chance of getting a hit is so much higher than average. Just as Joe Airball cannot match Larry Bird for runs of baskets, Joe’s cousin Bill Ofer, with a lifetime batting average of .184, will never have a streak to match DiMaggio’s with a lifetime average of .325. The statistics show something else, and something fascinating: There is no “causality of circumstance,” no “extra” that the great can draw from the soul of their valor to extend a streak beyond the ordinary expectation of coin-tossing models for a series of unconnected events, each occurring with a characteristic probability for that particular player. Good players have higher characteristic probabilities, hence longer streaks.