Skills in Predicting Past History
We can discuss this point from different angles. Experts call one manifestation of such denigration of history historical determinism. In a nutshell we think that we would know when history is made; we believe that people who, say, witnessed the stock market crash of 1929 knew then that they lived an acute historical event and that, should these events repeat themselves, they too would know about such facts. Life for us is made to resemble an adventure movie, as we know ahead of time that something big is about to happen. It is hard to imagine that people who witnessed history did not know at the time how important the moment was. Somehow all respect we may have for history does not translate well into our treatment of the present.
Jean-Patrice of the last chapter was abruptly replaced by an interesting civil servant type who had never been involved in the randomness professions. He just went to the right civil servant schools where people learn to write reports and had some senior managerial position in the institution. As is typical with subjectively assessed positions he tried to make his predecessor look bad: Jean-Patrice was deemed sloppy and unprofessional. The civil servant’s first undertaking was to run a formal analysis of our transactions; he found that we traded a little too much, incurring very large back office expenditure. He analyzed a large segment of foreign exchange traders’ transactions, then wrote a report explaining that only close to 1% of these transactions generated significant profits; the rest generated either losses or small profits. He was shocked that the traders did not do more of the winners and less of the losers. It was obvious to him that we needed to comply with these instructions immediately. If we just doubled the winners, the results for the institution would be so great. How come you highly paid traders did not think about it before?
Things are always obvious after the fact. The civil servant was a very intelligent person, and this mistake is much more prevalent than one would think. It has to do with the way our mind handles historical information. When you look at the past, the past will always be deterministic, since only one single observation took place. Our mind will interpret most events not with the preceding ones in mind, but the following ones. Imagine taking a test knowing the answer. While we know that history flows forward, it is difficult to realize that we envision it backward. Why is it so? We will discuss the point in Chapter 11 but here is a possible explanation: Our minds are not quite designed to understand how the world works, but, rather, to get out of trouble rapidly and have progeny. If they were made for us to understand things, then we would have a machine in it that would run the past history as in a VCR, with a correct chronology, and it would slow us down so much that we would have trouble operating. Psychologists call this overestimation of what one knew at the time of the event due to subsequent information the hindsight bias, the “I knew it all along” effect.
Now the civil servant called the trades that ended up as losers “gross mistakes,” just like journalists call decisions that end up costing a candidate his election a “mistake.” I will repeat this point until I get hoarse: A mistake is not something to be determined after the fact, but in the light of the information until that point.
A more vicious effect of such hindsight bias is that those who are very good at predicting the past will think of themselves as good at predicting the future, and feel confident about their ability to do so. This is why events like those of September 11, 2001, never teach us that we live in a world where important events are not predictable—even the Twin Towers’ collapse appears to have been predictable then.
My Solon
I have another reason to be obsessed with Solon’s warning. I hark back to the very same strip of land in the Eastern Mediterranean where the story took place. My ancestors experienced bouts of extreme opulence and embarrassing penury over the course of a single generation, with abrupt regressions that people around me who have the memory of steady and linear betterment do not think feasible (at least not at the time of writing). Those around me either have (so far) had few family setbacks (except for the Great Depression) or, more generally, are not suffused with enough sense of history to reflect backward. For people of my background, Eastern Mediterranean Greek Orthodox and invaded Eastern Roman citizens, it was as if our soul had been wired with the remembrance of that sad spring day circa 500 years ago when Constantinople, under the invading Turks, fell out of history, leaving us the lost subjects of a dead empire, very prosperous minorities in an Islamic world—but with an extremely fragile wealth. Moreover, I vividly remember the image of my own dignified grandfather, a former deputy prime minister and son of a deputy prime minister (whom I never saw without a suit), residing in a nondescript apartment in Athens, his estate having been blown up during the Lebanese civil war. Incidentally, having experienced the ravages of war, I find undignified impoverishment far harsher than physical danger (somehow dying in full dignity appears to me far preferable to living a janitorial life, which is one of the reasons I dislike financial risks far more than physical ones). I am certain that Croesus worried more about the loss of his Kingdom than the perils to his life.
There is an important and nontrivial aspect of historical thinking, perhaps more applicable to the markets than anything else: Unlike many “hard” sciences, history cannot lend itself to experimentation. But somehow, overall, history is potent enough to deliver, on time, in the medium to long run, most of the possible scenarios, and to eventually bury the bad guy. Bad trades catch up with you, it is frequently said in the markets. Mathematicians of probability give that a fancy name: ergodicity. It means, roughly, that (under certain conditions) very long sample paths would end up resembling each other. The properties of a very, very long sample path would be similar to the Monte Carlo properties of an average of shorter ones. The janitor in Chapter 1 who won the lottery, if he lived one thousand years, cannot be expected to win more lotteries. Those who were unlucky in life in spite of their skills would eventually rise. The lucky fool might have benefited from some luck in life; over the longer run he would slowly converge to the state of a less-lucky idiot. Each one would revert to his long-term properties.
DISTILLED THINKING ON YOUR PALMPILOT
Breaking News
The journalist, my bête noire, entered this book with George Will dealing with random outcomes. In the next step I will show how my Monte Carlo toy taught me to favor distilled thinking, by which I mean the thinking based on information around us that is stripped of meaningless but diverting clutter. For the difference between noise and information, the topic of this book (noise has more randomness) has an analog: that between journalism and history. To be competent, a journalist should view matters like a historian, and play down the value of the information he is providing, such as by saying: “Today the market went up, but this information is not too relevant as it emanates mostly from noise.” He would certainly lose his job by trivializing the value of the information in his hands. Not only is it difficult for the journalist to think more like a historian, but it is, alas, the historian who is becoming more like the journalist.
For an idea, age is beauty (it is premature to discuss the mathematics of the point).The applicability of Solon’s warning to a life in randomness, in contrast with the exact opposite message delivered by the prevailing media-soaked culture, reinforces my instinct to value distilled thought over newer thinking, regardless of its apparent sophistication—another reason to accumulate the hoary volumes by my bedside (I confess that the only news items I currently read are the far more interesting upscale social gossip stories found in Tatler, Paris Match, and Vanity Fair—in addition to The Economist). Aside from the decorum of ancient thought as opposed to the coarseness of fresh ink, I have spent some time phrasing the idea in the mathematics of evolutionary arguments and conditional probability. For an idea to have survived so long across so many cycles is indicative of its relative fitness. Noise, at least some noise, was filtered out. Mathematically, progress means that some new information is better than p
ast information, not that the average of new information will supplant past information, which means that it is optimal for someone, when in doubt, to systematically reject the new idea, information, or method. Clearly and shockingly, always. Why?
The argument in favor of “new things” and even more “new new things” goes as follows: Look at the dramatic changes that have been brought about by the arrival of new technologies, such as the automobile, the airplane, the telephone, and the personal computer. Middlebrow inference (inference stripped of probabilistic thinking) would lead one to believe that all new technologies and inventions would likewise revolutionize our lives. But the answer is not so obvious: Here we only see and count the winners, to the exclusion of the losers (it is like saying that actors and writers are rich, ignoring the fact that actors are largely waiters—and lucky to be ones, for the less comely writers usually serve French fries at McDonald’s). Losers? The Saturday newspaper lists dozens of new patents of such items that can revolutionize our lives. People tend to infer that because some inventions have revolutionized our lives that inventions are good to endorse and we should favor the new over the old. I hold the opposite view. The opportunity cost of missing a “new new thing” like the airplane and the automobile is minuscule compared to the toxicity of all the garbage one has to go through to get to these jewels (assuming these have brought some improvement to our lives, which I frequently doubt).
Now the exact same argument applies to information. The problem with information is not that it is diverting and generally useless, but that it is toxic. We will examine the dubious value of the highly frequent news with a more technical discussion of signal filtering and observation frequency farther down. I will say here that such respect for the time-honored provides arguments to rule out any commerce with the babbling modern journalist and implies a minimal exposure to the media as a guiding principle for someone involved in decision making under uncertainty. If there is anything better than noise in the mass of “urgent” news pounding us, it would be like a needle in a haystack. People do not realize that the media is paid to get your attention. For a journalist, silence rarely surpasses any word.
On the rare occasions when I boarded the 6:42 train to New York I observed with amazement the hordes of depressed business commuters (who seemed to prefer to be elsewhere) studiously buried in The Wall Street Journal, apprised of the minutiae of companies that, at the time of writing now, are probably out of business. Indeed it is difficult to ascertain whether they seem depressed because they are reading the newspaper, or if depressive people tend to read the newspaper, or if people who are living outside their genetic habitat both read the newspaper and look sleepy and depressed. But while early on in my career such focus on noise would have offended me intellectually, as I would have deemed such information as too statistically insignificant for the derivation of any meaningful conclusion, I currently look at it with delight. I am happy to see such mass-scale idiotic decision making, prone to overreaction in their postperusal investment orders—in other words I currently see in the fact that people read such material an insurance for my continuing in the entertaining business of option trading against the fools of randomness. (It takes a huge investment in introspection to learn that the thirty or more hours spent “studying” the news last month neither had any predictive ability during your activities of that month nor did it impact your current knowledge of the world. This problem is similar to the weaknesses in our ability to correct for past errors: Like a health club membership taken out to satisfy a New Year’s resolution, people often think that it will surely be the next batch of news that will really make a difference to their understanding of things.)
Shiller Redux
Much of the thinking about the negative value of information on society in general was sparked by Robert Shiller. Not just in financial markets; but overall his 1981 paper may be the first mathematically formulated introspection on the manner in which society in general handles information. Shiller made his mark with his 1981 paper on the volatility of markets, where he determined that if a stock price is the estimated value of “something” (say the discounted cash flows from a corporation), then market prices are way too volatile in relation to tangible manifestations of that “something” (he used dividends as proxy). Prices swing more than the fundamentals they are supposed to reflect, they visibly overreact by being too high at times (when their price overshoots the good news or when they go up without any marked reason) or too low at others. The volatility differential between prices and information meant that something about “rational expectation” did not work. (Prices did not rationally reflect the long-term value of securities and were overshooting in either direction.) Markets had to be wrong. Shiller then pronounced markets to be not as efficient as established by financial theory (efficient markets meant, in a nutshell, that prices should adapt to all available information in such a way as to be totally unpredictable to us humans and prevent people from deriving profits). This conclusion set off calls by the religious orders of high finance for the destruction of the infidel who committed such apostasy. Interestingly, and by some strange coincidence, it is that very same Shiller who was trounced by George Will only one chapter ago.
The principal criticism against Shiller came from Robert C. Merton. The attacks were purely on methodological grounds (Shiller’s analysis was extremely rough; for instance, his using dividends in place of earnings was rather weak). Merton was also defending the official financial theory position that markets needed to be efficient and could not possibly deliver opportunities on a silver plate. Yet the same Robert C. Merton later introduced himself as the “founding partner” of a hedge fund that aimed at taking advantage of market inefficiencies. Setting aside the fact that Merton’s hedge fund blew up rather spectacularly from the black swan problem (with characteristic denial), his “founding” such a hedge fund requires, by implication, that he agrees with Shiller about the inefficiency of the market. The defender of the dogmas of modern finance and efficient markets started a fund that took advantage of market inefficiencies! It is as if the Pope converted to Islam.
Things are not getting any better these days. At the time of writing, news providers are offering all manner of updates, “breaking news” that can be delivered electronically in a wireless manner. The ratio of undistilled information to distilled is rising, saturating markets. The elder’s messages need not be delivered to you as imminent news.
This does not mean that all journalists are fooled by randomness noise providers: There are hordes of thoughtful journalists in the business (I would suggest London’s Anatole Kaletsky and New York’s Jim Grant and Alan Abelson as the underrated representatives of such a class among financial journalists; Gary Stix among scientific journalists); it is just that prominent media journalism is a thoughtless process of providing the noise that can capture people’s attention and there exists no mechanism for separating the two. As a matter of fact, smart journalists are often penalized. Like the lawyer in Chapter 11 who does not care about the truth, but about arguments that can sway a jury whose intellectual defects he knows intimately, journalism goes to what can capture our attention, with adequate sound bites. Again, my scholarly friends would wonder why I am getting emotional stating the obvious things about the journalists; the problem with my profession is that we depend on them for what information we need to obtain.
Gerontocracy
A preference for distilled thinking implies favoring old investors and traders, that is, investors who have been exposed to markets the longest, a matter that is counter to the common Wall Street practice of preferring those that have been the most profitable, and preferring the youngest whenever possible. I toyed with Monte Carlo simulations of heterogeneous populations of traders under a variety of regimes (closely resembling historical ones), and found a significant advantage in selecting aged traders, using as a selection criterion their cumulative years of experience rather than their absolute success (conditional on their ha
ving survived without blowing up). “Survival of the fittest,” a term so hackneyed in the investment media, does not seem to be properly understood: Under regime switching, as we will see in Chapter 5, it will be unclear who is actually the fittest, and those who will survive are not necessarily those who appear to be the fittest. Curiously, it will be the oldest, simply because older people have been exposed longer to the rare event and can be, convincingly, more resistant to it. I was amused to discover a similar evolutionary argument in mate selection that considers that women prefer (on balance) to mate with healthy older men over healthy younger ones, everything else being equal, as the former provide some evidence of better genes. Gray hair signals an enhanced ability to survive—conditional on having reached the gray hair stage, a man is likely to be more resistant to the vagaries of life. Curiously, life insurers in renaissance Italy reached the same conclusion, by charging the same insurance for a man in his twenties as they did for a man in his fifties, a sign that they had the same life expectancy; once a man crossed the forty-year mark, he had shown that very few ailments could harm him. We now proceed to a mathematical rephrasing of these arguments.
Fooled by Randomness Page 9