Fooled by Randomness

by Nassim Nicholas Taleb

  Someone would rationally say to Janet: “Go read this book Fooled by Randomness by one mathematical trader on the deformations of chance in life; it would give you a statistical sense of perspective and would accordingly make you feel better.” As an author, I would like to offer a panacea for $14.95, but I would rather say that in my best hopes it may provide an hour or so of solace. Janet may need something more drastic for relief. I have repeated that becoming more rational, or not feeling emotions of social slights, is not part of the human race, at least not with our current biology. There is no solace to be found from reasoning—as a trader I have learned something about these unfruitful efforts to reason against the grain. I would advise Janet to move out, and go live in some blue-collar neighborhood where they would feel less humiliated by their neighbors and rise in the pecking order beyond their probability of success. They could use the deformation in the opposite direction. If Janet cares about status, then I would even recommend some of these large housing blocks.

  DOUBLE SURVIVORSHIP BIASES

  More Experts

  I recently read a bestseller called The Millionaire Next Door, an extremely misleading (but almost enjoyable) book by two “experts,” in which the authors try to infer some attributes that are common to rich people. They examined a collection of currently wealthy people and found out that these are unlikely to lead lavish lives. They call such people the accumulators; persons ready to postpone consumption in order to amass funds. Most of the appeal of the book comes from the simple but counterintuitive fact that these are less likely to look like very rich people—it clearly costs money to look and behave rich, not to count the time demands of spending money. Leading prosperous lives is time-consuming—shopping for trendy clothes, becoming conversant in Bordeaux wines, getting to know the expensive restaurants. All these activities can put high demands on one’s time and divert the subject from what should be the real preoccupation, namely the accumulation of nominal (and paper) wealth. The moral of the book is that the wealthiest are to be found among those less suspected to be wealthy. On the other hand, those who act and look wealthy subject their net worth to such a drain that they inflict considerable and irreversible damage to their brokerage account.

  I will set aside the point that I see no special heroism in accumulating money, particularly if, in addition, the person is foolish enough to not even try to derive any tangible benefit from the wealth (aside from the pleasure of regularly counting the beans). I have no large desire to sacrifice much of my personal habits, intellectual pleasures, and personal standards in order to become a billionaire like Warren Buffett, and I certainly do not see the point of becoming one if I were to adopt Spartan (even miserly) habits and live in my starter house. Something about the praise lavished upon him for living in austerity while being so rich escapes me; if austerity is the end, he should become a monk or a social worker—we should remember that becoming rich is a purely selfish act, not a social one. The virtue of capitalism is that society can take advantage of people’s greed rather than their benevolence, but there is no need to, in addition, extol such greed as a moral (or intellectual) accomplishment (the reader can easily see that, aside from very few exceptions like George Soros, I am not impressed by people with money). Becoming rich is not directly a moral achievement, but that is not where the severe flaw in the book lies.

  As we saw, the heroes of The Millionaire Next Door are the accumulators, people who defer spending in order to invest. It is undeniable that such strategy might work; money spent bears no fruit (except the enjoyment of the spender). But the benefits promised in the book seem grossly overstated. A finer read of their thesis reveals that their sample includes a double dose of survivorship bias. In other words, it has two compounding flaws.

  Visibility Winners

  The first bias comes from the fact that the rich people selected for their sample are among the lucky monkeys on typewriters. The authors made no attempt to correct their statistics with the fact that they saw only the winners. They make no mention of the “accumulators” who have accumulated the wrong things (members of my family are experts on that; those who accumulated managed to accumulate currencies about to be devalued and stocks of companies that later went bust). Nowhere do we see a mention of the fact that some people were lucky enough to have invested in the winners; these people no doubt would make their way into the book. There is a way to take care of the bias: Lower the wealth of your average millionaire by, say, 50%, on the grounds that the bias causes the average net worth of the observed millionaire to be higher by such amount (it consists in adding the effect of the losers into the pot). It would certainly modify the conclusion.

  It’s a Bull Market

  As to the second, more serious flaw, I have already discussed the problem of induction. The story focuses on an unusual episode in history; buying its thesis implies accepting that the current returns in asset values are permanent (the sort of belief that prevailed before the great crash that started in 1929). Remember that asset prices have (still at the time of writing) witnessed the greatest bull market in history and that values did compound astronomically during the past two decades. A dollar invested in the average stock would have grown almost twenty-fold since 1982—and that is the average stock. The sample might include people who invested in stocks performing better than average. Virtually all of the subjects became rich from asset price inflation, in other words from the recent inflation in financial paper and assets that started in 1982. An investor who engaged in the same strategy during less august days for the market would certainly have a different story to tell. Imagine the book being written in 1982, after the prolonged erosion of the inflation-adjusted value of the stocks, or in 1935, after the loss of interest in the stock market.

  Or consider that the United States stock market is not the only investment vehicle. Consider the fate of those who, in place of spending their money buying expensive toys and paying for ski trips, bought Lebanese lira denominated Treasury bills (as my grandfather did), or junk bonds from Michael Milken (as many of my colleagues in the 1980s did). Go back in history and imagine the accumulator buying Russian Imperial bonds bearing the signature of Czar Nicholas II and trying to accumulate further by cashing them from the Soviet government, or Argentine real estate in the 1930s (as my great-grandfather did).

  The mistake of ignoring the survivorship bias is chronic, even (or perhaps especially) among professionals. How? Because we are trained to take advantage of the information that is lying in front of our eyes, ignoring the information that we do not see. At the time of writing, pension funds and insurance companies in the United States and in Europe somehow bought the argument that “in the long term equities always pay off 9%” and back it up with statistics. The statistics are right, but they are past history. My argument is that I can find you a security somewhere among the 40,000 available that went up twice that amount every year without fail. Should we put the social security money into it?

  A brief summing up at this point: I showed how we tend to mistake one realization among all possible random histories as the most representative one, forgetting that there may be others. In a nutshell, the survivorship bias implies that the highest performing realization will be the most visible. Why? Because the losers do not show up.

  A GURU’S OPINION

  The fund management industry is populated with gurus. Clearly, the field is randomness-laden and the guru is going to fall into a trap, particularly if he has no proper training in inference. At the time of writing, there is one such guru who developed the very unfortunate habit of writing books on the subject. Along with one of his peers, he computed the success of a “Robin Hood” policy of investing with the least successful manager in a given population of managers. It consists in switching down by taking money away from the winner and allocating it to the loser. This goes against the prevailing wisdom of investing with a winning manager and taking away money from a losing one. Doing so, their “paper strategy” (i.e., as in a Mo
nopoly game, not executed in real life) derived considerably higher returns than if they stuck to the winning manager. Their hypothetical example seemed to them to prove that one should not stay with the best manager, as we would be inclined to do, but rather switch to the worst manager, or at least such seems to be the point they were attempting to convey.

  Their analysis presents one severe hitch that any graduate student should be able to pinpoint at the first reading. Their sample only had survivors. They simply forgot to take into account the managers who went out of business. Such a sample includes managers that were operating during the simulation, and are still operating today. True, their sample included managers who did poorly, but only those managers who did poorly and recovered, without getting out of business. So it would be obvious that investing with those who fared poorly at some point but recovered (with the benefit of hindsight) would yield a positive return! Had they continued to fare poorly, they would be out of business and would not be included in the sample.

  How should one conduct the proper simulation? By taking a population of managers in existence, say, five years ago and running the simulation up to today. Clearly, the attributes of those who leave the population are biased toward failure; few successful people in such a lucrative business call it quits because of extreme success. Before we turn to a more technical presentation of these issues, one mention of the much idealized buzzword of optimism. Optimism, it is said, is predictive of success. Predictive? It can also be predictive of failure. Optimistic people certainly take more risks as they are overconfident about the odds; those who win show up among the rich and famous, others fail and disappear from the analyses. Sadly.

  Nine

  •

  IT IS EASIER TO BUY AND SELL THAN FRY AN EGG

  Some technical extensions of the survivorship bias. On the distribution of “coincidences” in life. It is preferable to be lucky than competent (but you can be caught). The birthday paradox. More charlatans (and more journalists). How the researcher with work ethics can find just about anything in data. On dogs not barking.

  This afternoon I have an appointment with my dentist (it will mostly consist in the dentist picking my brain on Brazilian bonds). I can state with a certain level of comfort that he knows something about teeth, particularly if I enter his office with a toothache and exit it with some form of relief. It will be difficult for someone who knows literally nothing about teeth to provide me with such relief, except if he is particularly lucky on that day—or has been very lucky in his life to become a dentist while not knowing anything about teeth. Looking at his diploma on the wall, I determine that the odds that he repeatedly gave correct answers to the exam questions and performed satisfactorily on a few thousand cavities before his graduation—out of plain randomness—are remarkably small.

  Later in the evening, I go to Carnegie Hall. I can say little about the pianist; I even forgot her unfamiliar foreign-sounding name. All I know is that she studied in some Muscovite conservatory. But I can expect to get some music out of the piano. It will be rare to have someone who performed brilliantly enough in the past to get to Carnegie Hall and now turns out to have benefited from luck alone. The expectation of having a fraud who will bang on the piano, producing only cacophonous sounds, is indeed low enough for me to rule it out completely.

  I was in London last Saturday. Saturdays in London are magical; bustling but without the mechanical industry of a weekday or the sad resignation of a Sunday. Without a wristwatch or a plan I found myself in front of my favorite carvings by Canova at the Victoria and Albert Museum. My professional bent immediately made me question whether randomness played a large role in the carving of these marble statues. The bodies were realistic reproductions of human figures, except that they were more harmonious and finely balanced than anything I have seen mother nature produce on its own (Ovid’s materiam superabat opus comes to mind). Could such finesse be a product of luck?

  I can practically make the same statement about anyone operating in the physical world, or in a business in which the degree of randomness is low. But there is a problem in anything related to the business world. I am bothered because tomorrow, unfortunately, I have an appointment with a fund manager seeking my help, and that of my friends, in finding investors. He has what he claims is a good track record. All I can infer is that he has learned to buy and sell. And it is harder to fry an egg than buy and sell. Well . . . the fact that he made money in the past may have some relevance, but not terribly so. This is not to say that it is always the case; there are some instances in which one can trust a track record, but, alas, there are not too many of these. As the reader now knows, the fund manager can expect to be heckled by me during the presentation, particularly if he does not exhibit the minimum of humility and self-doubt that I would expect from someone practicing randomness. I will probably bombard him with questions that he may not be prepared to answer, blinded by his past results. I will probably lecture him that Machiavelli ascribed to luck at least a 50% role in life (the rest was cunning and bravura), and that was before the creation of modern markets.

  In this chapter, I discuss some well-known counterintuitive properties of performance records and historical time series. The concept presented here is well-known for some of its variations under the names survivorship bias, data mining, data snooping, over-fitting, regression to the mean, etc., basically situations where the performance is exaggerated by the observer, owing to a misperception of the importance of randomness. Clearly, this concept has rather unsettling implications. It extends to more general situations where randomness may play a share, such as the choice of a medical treatment or the interpretation of coincidental events.

  When I am tempted to suggest a possible future contribution of financial research to science in general, I adduce the analysis of data mining and the study of survivorship biases. These have been refined in finance but can extend to all areas of scientific investigation. Why is finance so rich a field? Because it is one of the rare areas of investigation where we have plenty of information (in the form of abundant price series), but no ability to conduct experiments as in, say, physics. This dependence on past data brings about its salient defects.

  FOOLED BY NUMBERS

  Placebo Investors

  I have often been faced with questions of the sort: “Who do you think you are to tell me that I might have been plain lucky in my life?” Well, nobody really believes that he or she was lucky. My approach is that, with our Monte Carlo engine, we can manufacture purely random situations. We can do the exact opposite of conventional methods; in place of analyzing real people hunting for attributes we can create artificial ones with precisely known attributes. Thus we can manufacture situations that depend on pure, unadulterated luck, without the shadow of skills or whatever we have called nonluck in Table P.1. In other words, we can man-make pure nobodies to laugh at; they will be by design stripped of any shadow of ability (exactly like a placebo drug).

  We saw in Chapter 5 how people may survive owing to traits that momentarily fit the given structure of randomness. Here we take a far simpler situation where we know the structure of randomness; the first such exercise is a finessing of the old popular saying that even a broken clock is right twice a day. We will take it a bit further to show that statistics is a knife that cuts on both sides. Let us use the Monte Carlo generator introduced earlier and construct a population of 10,000 fictional investment managers (the generator is not terribly necessary since we can use a coin, or even do plain algebra, but it is considerably more illustrative—and fun). Assume that they each have a perfectly fair game; each one has a 50% probability of making $10,000 at the end of the year, and a 50% probability of losing $10,000. Let us introduce an additional restriction; once a manager has a single bad year, he is thrown out of the sample, good-bye and have a nice life. Thus we will operate like the legendary speculator George Soros who was said to tell his managers gathered in a room: “Half of you guys will be out by next year” (with an E
astern European accent). Like Soros, we have extremely high standards; we are looking only for managers with an unblemished record. We have no patience for low performers.

  The Monte Carlo generator will toss a coin; heads and the manager will make $10,000 over the year, tails and he will lose $10,000. We run it for the first year. At the end of the year, we expect 5,000 managers to be up $10,000 each, and 5,000 to be down $10,000. Now we run the game a second year. Again, we can expect 2,500 managers to be up two years in a row; another year, 1,250; a fourth one, 625; a fifth, 313. We have now, simply in a fair game, 313 managers who made money for five years in a row. Out of pure luck.

  Meanwhile if we throw one of these successful traders into the real world we would get very interesting and helpful comments on his remarkable style, his incisive mind, and the influences that helped him achieve such success. Some analysts may attribute his achievement to precise elements among his childhood experiences. His biographer will dwell on the wonderful role models provided by his parents; we would be supplied with black-and-white pictures in the middle of the book of a great mind in the making. And the following year, should he stop outperforming (recall that his odds of having a good year have stayed at 50%) they would start laying blame, finding fault with the relaxation in his work ethics, or his dissipated lifestyle. They will find something he did before when he was successful that he has subsequently stopped doing, and attribute his failure to that. The truth will be, however, that he simply ran out of luck.

  Nobody Has to Be Competent

  Let’s push the argument further to make it more interesting. We create a cohort that is composed exclusively of incompetent managers. We will define an incompetent manager as someone who has a negative expected return, the equivalent of the odds being stacked against him. We instruct the Monte Carlo generator now to draw from an urn. The urn has 100 balls, 45 black and 55 red. By drawing with replacement, the ratio of red to black balls will remain the same. If we draw a black ball, the manager will earn $10,000. If we draw a red ball, he will lose $10,000. The manager is thus expected to earn $10,000 with 45% probability, and lose $10,000 with 55%. On average, the manager will lose $1,000 each round—but only on average.