Against the Gods: The Remarkable Story of Risk

by Peter L. Bernstein

  ... [W[ho can pretend to have penetrated so deeply into the nature of the human mind or the wonderful structure of the body that in games which depend ... on the mental acuteness or physical agility of the players he would venture to predict when this or that player would win or lose?

  Jacob is drawing a crucial distinction between reality and abstraction in applying the laws of probability. For example, Paccioli's incomplete game of balla and the unfinished hypothetical World Series that we analyzed in the discussion of Pascal's Triangle bear no resemblance to real-world situations. In the real world, the contestants in a game of balla or in a World Series have differing "mental acuteness or physical agility," qualities that I ignored in the oversimplified examples of how to use probability to forecast outcomes. Pascal's Triangle can provide only hints about how such real-life games will turn out.

  The theory of probability can define the probabilities at the gaming casino or in a lottery-there is no need to spin the roulette wheel or count the lottery tickets to estimate the nature of the outcome-but in real life relevant information is essential. And the bother is that we never have all the information we would like. Nature has established patterns, but only for the most part. Theory, which abstracts from nature, is kinder: we either have the information we need or else we have no need for information. As I quoted Fischer Black as saying in the Introduction, the world looks neater from the precincts of MIT on the Charles River than from the hurly-burly of Wall Street by the Hudson.

  In our discussion of Paccioli's hypothetical game of balla and our imaginary World Series, the long-term records, the physical capabil ities, and the I.Q.s of the players were irrelevant. Even the nature of the game itself was irrelevant. Theory was a complete substitute for information.

  Real-life baseball fans, like aficionados of the stock market, assemble reams of statistics precisely because they need that information in order to reach judgments about capabilities among the players and the teams-or the outlook for the earning power of the companies trading on the stock exchange. And even with thousands of facts, the track record of the experts, in both athletics and finance, proves that their estimates of the probabilities of the final outcomes are open to doubt and uncertainty.

  Pascal's Triangle and all the early work in probability answered only one question: what is the probability of such-and-such an outcome? The answer to that question has limited value in most cases, because it leaves us with no sense of generality. What do we really know when we reckon that Player A has a 60% chance of winning a particular game of balla? Can that likelihood tell us whether he is skillful enough to win 60% of the time against Player B? Victory in one set of games is insufficient to confirm that expectation. How many times do Messrs. A and B have to play before we can be confident that A is the superior player? What does the outcome of this year's World Series tell us about the probability that the winning team is the best team all the time not just in that particular series? What does the high proportion of deaths from lung cancer among smokers signify about the chances that smoking will kill you before your time? What does the death of an elephant reveal about the value of going to an air-raid shelter?

  But real-life situations often require us to measure probability in precisely this fashion-from sample to universe. In only rare cases does life replicate games of chance, for which we can determine the probability of an outcome before an event even occurs-a priori, as Jacob Bernoulli puts it. In most instances, we have to estimate probabilities from what happened after the fact-a posteriori. The very notion of a posteriori implies experimentation and changing degrees of belief. There were seven million people in Moscow, but after one elephant was killed by a Nazi bomb, the professor decided the time had come to go to the air-raid shelter.

  Jacob Bernoulli's contribution to the problem of developing probabilities from limited amounts of real-life information was twofold. First, he defined the problem in this fashion before anyone else had even recognized the need for a definition. Second, he suggested a solution that demands only one requirement. We must assume that "under similar conditions, the occurrence (or non-occurrence) of an event in the future will follow the same pattern as was observed in the past."5

  This is a giant assumption. Jacob may have complained that in real life there are too few cases in which the information is so complete that we can use the simple rules of probability to predict the outcome. But he admits that an estimate of probabilities after the fact also is impossible unless we can assume that the past is a reliable guide to the future. The difficulty of that assignment requires no elaboration.

  The past, or whatever data we choose to analyze, is only a fragment of reality. That fragmentary quality is crucial in going from data to a generalization. We never have all the information we need (or can afford to acquire) to achieve the same confidence with which we know, beyond a shadow of a doubt, that a die has six sides, each with a different number, or that a European roulette wheel has 37 slots (American wheels have 38 slots), again each with a different number. Reality is a series of connected events, each dependent on another, radically different from games of chance in which the outcome of any single throw has zero influence on the outcome of the next throw. Games of chance reduce everything to a hard number, but in real life we use such measures as "a little," "a lot," or "not too much, please" much more often than we use a precise quantitative measure.

  Jacob Bernoulli unwittingly defined the agenda for the remainder of this book. From this point forward, the debate over managing risk will converge on the uses of his three requisite assumptions-full information, independent trials, and the relevance of quantitative valuation. The relevance of these assumptions is critical in determining how successfully we can apply measurement and information to predict the future. Indeed, Jacob's assumptions shape the way we view the past itself. after the fact, can we explain what happened, or must we ascribe the event to just plain luck (which is merely another way of saying we are unable to explain what happened)?

  Despite all the obstacles, practicality demands that we assume, sometimes explicitly but more often implicitly, that Jacob's necessary conditions are met, even when we know full well that reality differs from the ideal case. Our answers may be sloppy, but the methodology developed by Jacob Bernoulli and the other mathematicians mentioned in this chapter provides us with a powerful set of tools for developing probabilities of future outcomes on the basis of the limited data provided by the past.

  Jacob Bernoulli's theorem for calculating probabilities a posteriori is known as the Law of Large Numbers. Contrary to the popular view, this law does not provide a method for validating observed facts, which are only an incomplete representation of the whole truth. Nor does it say that an increasing number of observations will increase the probability that what you see is what you are going to get. The law is not a design for improving the quality of empirical tests: Jacob took Leibniz's advice to heart and rejected his original idea of finding firm answers by means of empirical tests.

  Jacob was searching for a different probability. Suppose you toss a coin over and over. The Law of Large Numbers does not tell you that the average of your throws will approach 50% as you increase the number of throws; simple mathematics can tell you that, sparing you the tedious business of tossing the coin over and over. Rather, the law states that increasing the number of throws will correspondingly increase the probability that the ratio of heads thrown to total throws will vary from 50% by less than some stated amount, no matter how small. The word "vary" is what matters. The search is not for the true mean of 50% but for the probability that the error between the observed average and the true average will be less than, say, 2%-in other words, that increasing the number of throws will increase the probability that the observed average will fall within 2% of the true average.

  That does not mean that there will be no error after an infinite number of throws; Jacob explicitly excludes that case. Nor does it mean that the errors will of necessity become small enough to ignore.
All the law tells us is that the average of a large number of throws will be more likely than the average of a small number of throws to differfrom the true average by less than some stated amount. And there will always be a possibility that the observed result will differ from the true average by a larger amount than the specified bound. Seven million people in Moscow were apparently not enough to satisfy the professor of statistics.

  The Law of Large Numbers is not the same thing as the Law of Averages. Mathematics tells us that the probability of heads coming up on any individual coin toss is 50%-but the outcome of each toss is independent of all the others. It is neither influenced by previous tosses nor does it influence future tosses. Consequently, the Law of Large Numbers cannot promise that the probability of heads will rise above 50% on any single toss if the first hundred, or million, tosses happen to come up only 40% heads. There is nothing in the Law of Large Numbers that promises to bail you out when you are caught in a losing streak.

  To illustrate his Law of Large Numbers, Jacob hypothesized a jar filled with 3000 white pebbles and 2000 black pebbles, a device that has been a favorite of probability theorists and inventors of mind-twisting mathematical puzzles ever since. He stipulates that we must not know how many pebbles there are of each color. We draw an increasing number of pebbles from the jar, carefully noting the color of each pebble before returning it to the jar. If drawing more and more pebbles can finally give us "moral certainty"-that is, certainty as a practical matter rather than absolute certainty-that the ratio is 3:2, Jacob concludes that "we can determine the number of instances a posteriori with almost as great accuracy as if they were know to us a priori."6 His calculations indicate that 25,550 drawings from the jar would suffice to show, with a chance exceeding 1000/1001, that the result would be within 2% of the true ratio of 3:2. That's moral certainty for you.

  Jacob does not use the expression "moral certainty" lightly. He derives it from his definition of probability, which he draws from earlier work by Leibniz. "Probability," he declares, "is degree of certainty and differs from absolute certainty as the part differs from the whole."

  But Jacob moves beyond Leibniz in considering what "certainty" means. It is our individual judgments of certainty that attract Jacob's attention, and a condition of moral certainty exists when we are almost completely certain. When Leibniz introduced the concept, he had defined it as "infinitely probable." Jacob himself is satisfied that 1000/1001 is close enough, but he is willing to be flexible: "It would be useful if the magistrates set up fixed limits for moral certainty."8

  Jacob is triumphant. Now, he declares, we can make a prediction about any uncertain quantity that will be just as scientific as the predictions made in games of chance. He has elevated probability from the world of theory to the world of reality:

  If, instead of the jar, for instance, we take the atmosphere or the human body, which conceal within themselves a multitude of the most varied processes or diseases, just as the jar conceals the pebbles, then for these also we shall be able to determine by observation how much more frequently one event will occur than another.9

  Yet Jacob appears to have had trouble with his jar of pebbles. His calculation that 25,550 trials would be necessary to establish moral certainty must have struck him as an intolerably large number; the entire population of his home town of Basel at that time was less than 25,550. We must surmise that he was unable to figure out what to do next, for he ends his book right there. Nothing follows but a wistful comment about the difficulty of finding real-life cases in which all the observations meet the requirement that they be independent of one another:

  If thus all events through all eternity could be repeated, one would find that everything in the world happens from definite causes and according to definite rules, and that we would be forced to assume amongst the most apparently fortuitous things a certain necessity, or, so to say, FATE.1o

  Nevertheless, Jacob's jar of pebbles deserves the immortality it has earned. Those pebbles became the vehicle for the first attempt to measure uncertainty-indeed, to define it-and to calculate the probability that an empirically determined number is close to a true value even when the true value is an unknown.

  Jacob Bernoulli died in 1705. His nephew Nicolaus-Nicolaus the Slow-continued to work on Uncle Jacob's efforts to derive future probabilities form known observations even while he was inching along toward the completion of Ars Conjectandi. Nicolaus's results were published in 1713, the same year in which Jacob's book finally appeared.

  Jacob had started with the probability that the error between an observed value and the true value would fall within some specified bound; he then went on to calculate the number of observations needed to raise the probability to that amount. Nicolaus tried to turn his uncle's version of probability around. Taking the number of observations as given, he then calculated the probability that they would fall within the specified bound. He used an example in which he assumed that the ratio of male to female births was 18:17. With, say, a total of 14,000 births, the expected number of male births would be 7,200. He then calculated that the odds are at least 43.58-to-1 that the actual number of male births would fall between 7,200 + 163 and 7,200 - 163, or between 7,363 and 7,037.

  In 1718, Nicolaus invited a French mathematician named Abraham de Moivre to join him in his research, but de Moivre turned him down: "I wish I were capable of... applying the Doctrine of Chances to Oeconomical and Political Uses [but] I willingly resign my share of that task to better Hands."11 Nevertheless, de Moivre's response to Nicolaus reveals that the uses of probability and forecasting had come a long way in just a few years.

  De Moivre had been born in 1667-thirteen years after Jacob Bernoulli-as a Protestant in a France that was increasingly hostile to anyone who was not Catholic. 12 In 1685, when de Moivre was 18 years old, King Louis XIV revoked the Edict of Nantes, which had been promulgated under the Protestant-born King Henri IV in 1598 to give Protestants, known as Huguenots, equal political rights with Catholics. After the revocation, exercise of the reformed religion was forbidden, children had to be educated as Catholics, and emigration was prohibited. De Moivre was imprisoned for over two years for his beliefs. Hating France and everything to do with it, he managed to flee to London in 1688, where the Glorious Revolution had just banished the last vestiges of official Catholicism. He never returned to his native country.

  De Moivre led a gloomy, frustrating life in England. Despite many efforts, he never managed to land a proper academic position. He supported himself by tutoring in mathematics and by acting as a consultant to gamblers and insurance brokers on applications of probability theory. For that purpose, he maintained an informal office at Slaughter's Coffee House in St. Martin's Lane, where he went most afternoons after his tutoring chores were over. Although he and Newton were friends, and although he was elected to the Royal Society when he was only thirty, he remained a bitter, introspective, antisocial man. He died in 1754, blind and poverty-stricken, at the age of 87.

  In 1725, de Moivre had published a work titled Annuities upon Lives, which included an analysis of Halley's tables on life and death in Breslaw. Though the book was primarily a work in mathematics, it suggested important questions related to the puzzles that the Bernoullis were trying to resolve and that de Moivre would later explore in great detail.

  Stephen Stigler, a historian of statistics, offers an interesting example of the possibilities raised by de Moivre's work in annuities. Halley's table showed that, of 346 men aged fifty in Breslaw, only 142, or 41%, survived to age seventy. That was only a small sample. To what extent could we use the result to generalize about the life expectancy of men fifty years old? De Moivre could not use these numbers to determine the probability that a man of fifty had a less than 50% chance of dying by age seventy, but he would be able to answer this question: "If the true chance were 1/2, what is the probability a ratio as small as 142/346 or smaller should occur?"

  De Moivre's first direct venture into the subject of probabi
lity was a work titled De Mensura Sortis (literally, On the Measurement of Lots). This work was first published in 1711 in an issue of Philosophical Transactions, the publication of the Royal Society. In 1718, de Moivre issued a greatly expanded English edition titled The Doctrine of Chances, which he dedicated to his good friend Isaac Newton. The book was a great success and went through two further editions in 1738 and 1756. Newton was sufficiently impressed to tell his students on occasion, "Go to Mr. de Moivre; he knows these things better than I do." De Mensura Sortis is probably the first work that explicitly defines risk as chance of loss: "The Risk of losing any sum is the reverse of Expectation; and the true measure of it is, the product of the Sum adventured multiplied by the Probability of the Loss."

  In 1730, de Moivre finally turned to Nicolaus Bernoulli's project to ascertain how well a sample of facts represented the true universe from which the sample was drawn. He published his complete solution in 1733 and included it in the second and third editions of Doctrine of Chances. He begins by acknowledging that Jacob and Nicolaus Bernoulli "have shewn very great skill .... [Y]et some things were farther required." In particular, the approach taken by the Bernoullis appeared "so laborious, and of so great difficulty, that few people have undertaken the task."

  The need for 25,550 trials was clearly an obstacle. Even if, as James Newman has suggested, Jacob Bernoulli had been willing to settle for the "immoral certainty" of an even bet-probability of 50/100-that the result would be within 2% of the true ratio of 3:2, 8,400 drawings would be needed. Jacob's selection of a probability of 1000/1001 is in itself a curiosity by today's standards, when most statisticians accept odds of 1 in 20 as sufficient evidence that a result is significant (today's lingo for moral certainty) rather than due to mere chance.