The Fractalist

by Benoit Mandelbrot

  Unfortunately, my careful tests that should have blown up Bachelier in 1963 failed. The economics profession decided that my work was too complicated and too unfamiliar. The departure it represented and further threatened was hard to develop and sell. It seemed far easier to continue with an endless stream of “fixes.” What was I to do? I moved to an altogether different set of “priority interests,” with only episodic returns to price variation. The 1900 theory of finance that I had discredited has persisted, attracting many young mathematicians and scientists, depleting the fields they come from.

  And then, perhaps a bit later than I expected—in 2008—the market did what it was bound to do: it crashed.

  Kepler Versus Ptolemy

  Bachelier’s 1900 and my 1963 models of price variation were the first two to be put forward and are the stars of the events that will unfold shortly. Is the topic doomed to be presented forever in terms of this contrast? I am afraid it is and—in all modesty—would like to explain why, by comparison with a key event in science: the replacement of the ancients’ faulty model of the motion of planets by Kepler’s ellipses. Ptolemy’s model stated that the planets revolved in a circular orbit around Earth. However, he regularly had to amend it when he observed anomalies. This belief was widely held until the 1600s, when Kepler proved that the planets revolved around the sun in an elliptical orbit.

  Bachelier had assumed that price changes follow an ancient frequency curve—a familiar one—called Gaussian. Its key property is that large deviations from the norm have an absolutely tiny probability and therefore do not matter. In the following price fluctuation charts, the top chart is Bachelier’s model, showing that most changes are small. The middle is a real price chart for IBM’s stock price, showing some outliers and far greater fluctuation than Bachelier’s model. The bottom chart uses my computer-generated multifractal model, showing that it stands up to actual records of changes in financial prices.

  (Illustration Credit 22.2)

  I provided the data with an intellectual home—not surprisingly, one that hardly any practical person knew of at that time. Indeed, Paul Lévy’s teachings had familiarized me with a curio he called stability and I prefer to call Lévy stability. Hence, I could identify the behavior as a characteristic of price change. Working at IBM, I had access to a computer center where—laboriously—Lévy stable densities could actually be calculated for the first time.

  In the case of cotton—with no fiddling—the fit was striking. My first work in finance had brought together two domains of knowledge far removed from one another. Surprisingly, the stable distribution fit every detail of the data—in particular, a symmetry in the distribution that earlier examinations had missed.

  It is often asserted that if one adds many statistical quantities, even the largest is conveniently negligible in comparison to the sum. However, the contrary had long been known to happen—but only in cases with which practical statisticians need not be concerned. A loud opponent of mine repeatedly claimed that those cases were “improper,” but this view led him astray. In fact, those cases were well known to experts, but were viewed as belonging to irrelevant pure mathematics. By bringing them into all-too-practical finance, I imposed and argued for a deep distinction between “mild” and “wild” states of randomness of chance. To the best of my knowledge, all past work on prices did not conceive of this wildness and had confidently relied on the reality being ruled by randomness that was “proper,” therefore mild.

  The three states of chance—wild, mild, and slow—can be compared to the three states of matter. Are not solid and gas separated by liquid? Absolutely. In my view, the same is true of chance—the counterpart of liquid being “slow” randomness. And liquids happen to be enormously more complicated to study.

  A Backhanded Compliment?

  There can be little doubt that Mandelbrot’s hypotheses are the most revolutionary development in the theory of speculative prices since Bachelier’s initial work [of 1900. His] papers force us to face up in a substantial way to those uncomfortable empirical observations that there is little doubt most of us have had to sweep under the carpet up to now. With determination and passion he has marshaled as an integral part of his argument evidence of a more complicated and much more disturbing view of the economic world than economists have hitherto endorsed.… Mandelbrot, like Prime Minister Winston Churchill before him, promises us not utopia, but blood, sweat, toil and tears. If [he] is right, almost all our statistical tools [and] past econometric work [are] meaningless.…

  Surely, before consigning centuries of work to the ash pile, we should like to have some assurance that all our work is truly useless.

  This mixture of faint praise and insidious attack is from a text by the economist Paul H. Cootner (1930–78). I first saw it in December 1962. Technically, this attack would have been easy to answer, but politically, this and others like it led me to conclude that business economists’ blind loyalty to Bachelier’s 1900 theory was too ingrained to be overcome. So I simply stepped aside. No matter what I might say or do, heavy criticism was bound to follow. Leaving French academia for an American industrial laboratory—a colossal gamble—had proved I was prepared to take controversial stands. At that time, however, the last thing I could afford was to be a Mr. No. So I swallowed hard and moved on. Soon enough, I would tackle another, less political problem: turbulence in fluids and its extension to large scales, commonly called the weather.

  Financial orthodoxy is founded on two critical assumptions in Bachelier’s key model: price changes are statistically independent, and they are normally distributed. The facts, as I vehemently argued in the 1960s and many economists now acknowledge, show otherwise. First, price changes are not independent of each other. Research over the past few decades, by me and then by others, reveals that many financial price series have a “memory” of sorts. If prices take a big leap up or down now, there is a measurably greater likelihood that they will move just as violently the next day. It is not a well-behaved, predictable pattern of the kind economists prefer—not, say, the periodic up-and-down procession from boom to bust with which textbooks trace the standard business cycle. As my later work showed, it is a more complex, long-term memory—one that can be analyzed fractally. Second, the distribution of price changes is not “normal.” Conventional theory says that if you measure the changes from one day, hour, or month to the next, the vast majority of the changes should be very small, with only a few days exhibiting big changes—the “outliers” on the standard bell curve typically used to graph them. In fact, there are many more big changes than standard theory says there should be—many more days when prices crash or soar.

  Before sweeping all that “uncomfortable” data under the carpet, Cootner should have examined how much information he was destroying. This was hard to illustrate in the 1960s, but now it has become very easy. Next to the plot of an actual price index, test what would happen, at every moment, if Cootner had “swept under the carpet” the x largest price drops over x days. The index would more or less double. In other words, the few largest increments that Cootner discarded and the increments he preserved are equally important. The figure above dramatically illustrates this. The bottom line plots the actual daily S&P 500 price index from 1990 to 2005, and the top plots the same data without the ten largest daily price moves.

  (Illustration Credit 22.3)

  A second key misstep of Bachelier was even more serious, and correcting it took years. All along, everybody who cared about price variation knew about business cycles. The analysis I carried out in 1962–63 mixed together the phases of low and high price variability. A more realistic model must dig deeper into the data. A common way out introduces the concept of business cycles and assumes that different phases of a cycle follow different rules. Unfortunately, cycle timing has always been mysterious, unreliable, and discernible only long after the fact.

  The first challenges to Bachelier were entirely my work, and for decades continued to develop largely i
n my hands. In due time, I was able to claim that variations in financial prices can be accounted for by a model derived from my work in fractal geometry. An observer cannot tell which of the data concerns prices that change from week to week, day to day, or hour to hour. This quality defines the price charts as self-affine fractal curves and makes available many powerful tools of mathematical analysis. Fractals—or their later elaboration, multifractals—do not claim to predict the future with certainty. But they do create a more realistic picture of market risks than does observation alone. Eventually, and most unexpectedly, I combined that work with my older theory of word frequencies, and this led me to the fractal geometry of roughness.

  Does My 1963 Model Always Apply?

  My model does not always apply. In my first printed article on price variation, “The Variation of Certain Speculative Prices,” I underlined the meaning of “certain” by pointing out areas where the issue was not closed. In later life, I was often asked to talk about prices. Once, when my key interest was the alternance in fluids between laminar and turbulent zones, I was showing a trusty old slide on prices when—quite suddenly—I perceived an uncanny resemblance to a feature of turbulence called intermittence. Not a completely new idea, perhaps, insofar as the banker John Pierpont Morgan (1837–1913) had reputedly claimed the market was as fickle as the weather. While lecturing on, I told myself, How silly you have been. Don’t mention your thought today, but make sure to look into it when you can. In due time, I did.

  Will Training a Grad Student in Finance Lead to a Job in Chicago?

  Eugene F. Fama, a student at the University of Chicago Graduate School of Business, often visited me. More important, I met his Chicago adviser, Merton Miller (1923–2000), who convinced his business school colleagues to hire me. First, he brought me to Chicago, with Aliette. Although there was heavy snow all around, my lecture was mobbed—and a big party followed, given by the dean, George Shultz.

  It was clear that an offer had already been arranged and Chicago simply wanted to see what they were purchasing. At some point in my visit, my total lack of experience with U.S. universities led me to take a characteristically costly and foolish step. A written offer could always be “traded” for advantage in some other place—at IBM, in the University of Chicago, or elsewhere—but I had no mentor. Talking with Shultz made me realize that all they knew about me was an IBM report. They had not even looked up my vita and so did not know that in addition to being a freshly anointed pioneer in finance, I had extensive other interests. Standing at this critical fork in the road, I unwisely enlightened them. Shultz was very warm and commented that it was wonderful that one salary was going to get them several different professors.

  Back in Boston, the telephone rang: George Shultz on the line. He thanked me for that beautiful lecture, et cetera, et cetera, then came to the point. The offer was withdrawn. Really? He had rushed to ask several other departments whether they would share my salary. The answer was always no—they did not even know me. This left him with a problem that was the bane of my life: my tendency to cross scientific disciplines. He feared that my interests might move out of economics as smoothly and unexpectedly as they had moved in. This represented a risk he would not take.

  He was disappointed that his diplomatic skills had not been sufficient. He also reassured me that my thinking in economics would be well represented, because Eugene Fama was going to join the faculty.

  There is irony in this. This was the same Fama who, in 1964, submitted a thesis subtitled “A Test of Mandelbrot’s Stable Paretian Hypothesis.” He believed that successive price changes were statistically independent. I had to convince him that I had never claimed independence and that he was in fact testing a much weaker hypothesis—the one that was first expounded in Bachelier’s 1900 Ph.D. thesis and had become known as the martingale hypothesis. Fama conceded, corrected his earlier assertions, replaced the mysterious label “martingale” with “efficient market,” and built his career on becoming its champion. This hypothesis is convenient indeed, and it is, on occasion, useful as a first approximation or illustration. But on more careful examination, it failed to be verified—and for being its herald, Fama should receive neither blame nor credit.

  Continuing to follow my lead, he supervised several excellent Ph.D. dissertations. Next he returned to the fold and had a brilliant career as one of the leaders of his profession’s backlash to the strictest Bachelier orthodoxy—suitably “nostrified” by a new vocabulary. Naturally, the University of Chicago soon stopped inviting me.

  Shultz’s path and mine crossed again once at a gathering of U.S. residents holding the Légion d’Honneur. He remembered the episode and thought it had ended well. Probably the diplomat was being diplomatic. Shultz went on to run Bechtel, a huge construction corporation in California, then became, successively, Nixon’s director of the Office of Management and Budget, secretary of labor, and secretary of the Treasury. Later, as President Reagan’s secretary of state, he brought his diplomatic skills to the world stage.

  23

  On to Fractals: Through IBM, Harvard, MIT, and Yale via Economics, Engineering, Mathematics, and Physics, 1963–64

  YES, I HAVE BEEN WARNED. This chapter’s title seems to make no sense. How can it possibly reflect reality? Surprisingly, it does approximate this period of my life. All too often while waiting to deliver a lecture, I’ve heard the host conclude the customary introduction by wondering how I could conceivably exist. In fact, this chapter’s title is only a raw outline. Numerous additional fields I visited also differ deeply yet share a key feature that to me matters more than any other: roughness.

  Instead of the gritty term “engineering,” why don’t I use the more genteel “applied science”? In part, because one word is better than two. But largely because I want to make a point. The phenomena I have studied are elusive and not yet covered by any proper quantitative science—pure or applied. Think of a distant past. Water mills came long before the applied science of fluid mechanics; heat engines came before the applied science of heat. Stock exchanges arose before any theory, and no theory existed to which my work on finance could be “applied.” My ambition was more realistic—that is, more limited—yet essential. I wanted to provide a consistently more faithful description of known facts—and hence help financial engineering out of its dismal and harmful state. The same goes for the developments that will be described in this chapter: no existing body of science could assist them.

  What I have just said explains why I did not fear moving into a variety of problems of engineering. To master many applied sciences would be an idle dream—especially for an outsider like me—and is a process that would be unwise to rush.

  I led a complicated life in parallel to my quiet one at IBM—a life as a teacher or researcher who meandered from place to place or from one field to another. The fact that my life’s most productive season came late kept me in a constant hurry, and I could rarely take it easy. Before the fact, the path I was following seemed wild and impossible to manage; after the fact, it seems unavoidable. Halfway through each discovery—no sooner—I experienced a marvelous and exhilarating surprise.

  Had I come to IBM either when it was not yet ready or when it was already all too well organized, I might have returned to French academia. Few of my colleagues had my luck with an unplanned and truly extraordinary coincidence: that of an individual ready to experiment across many different fields and a corporation willing to trust that individual’s judgment. Essentially, all I had learned through my otherwise too long and too scattered wandering years gradually changed from a random burden on my memory to a very valuable asset in my work.

  I worked in the four fields of research in this chapter’s title—economics, engineering, mathematics, and physics—and I became involved with fractals in the arts. The first three are the departments I taught in as a visiting professor at Harvard. Of those three, nothing beats my impact on finance and mathematics. Physics—which I fear was least affected—re
warded my work most handsomely. My other work influenced rather small communities. A deep unity that had been present in my work all along was gradually revealed, then increased its presence and became my guide.

  Having worked in many fields but never wholly belonging to any, I consider myself an outlier. It does not hurt that the word “outlier” has an established technical meaning in statistics: it is an observation that is so very different from the norm that it may be due to accidental foreign contamination. A classic example concerns astronomical observations that were contaminated by cats residing in the observatory. Yes, cats walking across the observatory floor shook the telescope a bit, causing some orbits to be miscalculated. For two centuries, economists and statisticians have looked for good ways of preserving real data while eliminating would-be cats. To the contrary, I have found that the so-called outliers are essential in finance. In fact, a common thread of my work is that values far from the norm are the key to the underlying phenomenon. In many fields I remained far from the norm, so in that way too I am an outlier.

  Hydrology: The Biblical Joseph, Hurst, and Me

  There come seven years of great plenty throughout all the land of Egypt: And there shall arise after them seven years of famine. These words are found in the Bible. Look up Genesis 41:29–30 for the story of how Pharaoh had a dream and his high helper, Joseph, son of Jacob, interpreted that dream—and saved Egypt from a famine by storing enough grain for the lean years.

  An indomitable self-taught scholar who went on to graduate from Oxford, hydrologist Harold Edwin Hurst (1880–1978) interpreted that dream as reflecting a feature of the notorious variability of the waters of the Nile. Known as Abu Nil (Father of the Nile), Hurst remade himself into an expert on the Nile River Basin and became a champion of the need for the Aswan High Dam. He spent years searching modern data for a “signature” of Joseph’s interpretations. The topic being hot and potentially very costly, many experts were called in. In 1951, Hurst proposed a solution for optimum dam design based on what his research found. The experts opined that this formula by an undereducated author could not possibly hold.