Misbehaving: The Making of Behavioral Economics

by Richard H. Thaler

  No one takes the extreme version of this “no trade theorem” literally, but most financial economists agree, at least when pressed, that trading volume is surprisingly high. There is room for differences of opinion on price in a rational model, but it is hard to explain why shares would turn over at a rate of about 5% per month in a world of Econs. However, if you assume that some investors are overconfident, high trading volume emerges naturally. Jerry has no trouble doing the trade with Tom, because he thinks that he is smarter than Tom, and Tom thinks he’s smarter than Jerry. They happily trade, each feeling a twinge of guilt for taking advantage of his friend’s poor judgment.

  I find the overconfidence explanation of why we observe such high trading volume highly plausible, but it is also impossible to prove that it is right. Werner and I wanted to do something more convincing. We wanted to use a finding from psychology to predict something not previously known about financial markets and, even better, something that financial economists thought could not happen. Piece of cake.

  Our plan was to use a Kahneman and Tversky finding: that people are willing to make extreme forecasts based on flimsy data. In one of the pair’s classic experiments illustrating this point, subjects were asked to predict the grade point average (GPA) for a group of students based on a single fact about each one. There were two* conditions. In one condition, subjects were told the decile of the student’s GPA—that is, whether it fell in the top 10% (top decile, between the 90th and 100th percentile), the next 10% (between the 80th and 90th percentile), and so forth. The other group was not told anything about grades, but was instead given a decile score for each student on a test of “sense of humor.”

  Decile GPA is an excellent predictor of actual GPA, of course, so if you are told that Athena is in the top decile in GPA, you can reasonably predict that she has appropriately high grades, say 3.9 out of 4.0. But any correlation between sense of humor and GPA is likely to be weak, if it exists at all.

  If the subjects in Kahneman and Tversky’s experiment behaved rationally, those given percentile GPA would offer much more extreme (very high or low) predictions of actual GPA than those given measures of a test of sense of humor. Subjects who were told only about sense of humor should make forecasts that differ little from the average GPA at that school. In short, they shouldn’t let the sense of humor score influence their prediction much, if at all. As shown in figure 11, this did not happen. The forecasts based on sense of humor are nearly as extreme as the forecasts based on decile GPA. In fact, the predicted GPA for students who scored in the top decile in sense of humor was predicted to be the same as the GPA of those who were in the top decile based on GPA! One way to characterize this result is to say that the subjects overreacted to information about a student’s sense of humor.

  FIGURE 11

  Would investors behave the same way, responding to “ephemeral and non-significant” day-to-day information, as Keynes had asserted? And, if investors did overreact, how could we show it?

  Circumstantial evidence for overreaction already existed, namely the long-standing tradition of “value investing” pioneered by investment guru Benjamin Graham, author of the classic investment bibles Security Analysis, co-written with David Dodd and first published in 1934, and The Intelligent Investor, first published in 1949. Both books are still in print. Graham, like Keynes, was both a professional investor and a professor. He taught at Columbia University, where one of his students was the legendary investor Warren Buffett, who considers Graham his intellectual hero. Graham is often considered the father of “value investing,” in which the goal is to find securities that are priced below their intrinsic, long-run value. The trick is in knowing how to do this. When is a stock “cheap”? One of the simple measures that Graham advocated in order to decide whether a stock was cheap or expensive was the price/earnings ratio (P/E), the price per share divided by annual earnings per share. If the P/E ratio is high, investors are paying a lot per dollar of earnings, and implicitly, a high P/E ratio is a forecast that earnings will grow quickly to justify the current high price. If earnings fail to grow as quickly as anticipated, the price of the stock will fall. Conversely, for a stock with a low price/earnings ratio, the market is forecasting that earnings will remain low or even fall. If earnings rebound, or even remain stable, the price of the stock will rise.

  In the last edition of The Intelligent Investor written while Graham was alive (others have revised it since), he includes a simple table illustrating the efficacy of his approach. Starting in 1937, he took the thirty stocks included in the Dow Jones Industrial Average (some of the largest companies in America) and ranked them based on P/E. He then formed two portfolios—one of the ten stocks with the highest P/Es and the other of the ten stocks with the lowest P/Es—and showed that the “cheap” stocks outperformed the expensive group by an impressive margin. Over the period from 1937 to 1969, a $10,000 investment in the cheap stocks would have increased in value to $66,900, while the expensive stock portfolio would only have increased to $25,300. (Buying the entire original thirty-stock portfolio would have produced $44,000.) Implicitly, Graham was offering a kind of behavioral explanation for this finding. Cheap stocks were unpopular or out of favor, while expensive stocks were fashionable. By being a contrarian, Graham argued, you could beat the market, although not all the time. Graham noted that his strategy of buying the cheapest members of the Dow Jones Industrials would not have worked over an earlier period, 1917–33, and he cautioned that “Undervaluations caused by neglect or prejudice may persist for an inconveniently long time, and the same applies to inflated prices caused by overenthusiasm or artificial stimulants.” That advice was worth heeding during the technology bubble of the late 1990s, when value investing performed spectacularly badly, since the most expensive stocks, the Internet darlings, kept increasing in price, leaving those boring value stocks behind.

  Many in the investment community revered Benjamin Graham, but by the early 1980s most academic financial economists considered his work passé. A simple strategy of buying “cheap” stocks was obviously inconsistent with the Efficient Market Hypothesis, and Graham’s methods were hardly state of the art. The data for the returns on the various Dow portfolios had undoubtedly been constructed by hand. Now researchers had digitized data files such as CRSP for stock prices and COMPUSTAT, which collected financial accounting data. When these two data sources came together, much more comprehensive studies were possible, and results like Graham’s that used a small number of stocks over a relatively short time period were considered little more than anecdotes.

  It was not so much that anyone had refuted Graham’s claim that value investing worked; it was more that the efficient market theory of the 1970s said that value investing couldn’t work. But it did. Late that decade, accounting professor Sanjoy Basu published a thoroughly competent study of value investing that fully supported Graham’s strategy. However, in order to get such papers published at the time, one had to offer abject apologies for the results. Here is how Basu ended his paper: “In conclusion, the behavior of security prices over the fourteen-year period studied is, perhaps, not completely described by the efficient market hypothesis.” He stopped just short of saying “I am sorry.” Similarly, one of Eugene Fama’s students at the University of Chicago, Rolf Banz, discovered another anomalous finding, namely that portfolios of small firms outperformed portfolios of large firms. Here is his own apologetic conclusion in his paper published in 1981: “Given its longevity, it is not likely that it is due to a market inefficiency but it is rather evidence of a pricing model misspecification.” In other words, there must be something left out of the model because market efficiency cannot be wrong.

  An investor named David Dreman made bolder claims related to Graham. Dreman had founded his own investment company and had somehow stumbled onto the work of Kahneman and Tversky. He was the first person to suggest an explicitly psychological explanation for the value effect, one based on the tendency for people to extrapol
ate the recent past into the future. Dreman published his ideas in 1982 in a book aimed at a popular audience titled The New Contrarian Investment Strategy. Unlike Basu and Banz, he offered no apologies for his ideas, but because it was a book for nonspecialists it did not make much of an impression on the academic finance community. But Werner and I read the book and took notice.

  Following Dreman’s thinking led us to a plausible hypothesis. Suppose that the “P/E effect” is caused by overreaction: high P/E stocks (known as growth stocks because they are going to have to grow like crazy to justify their high prices) have gone up “too high” because investors have made overly optimistic forecasts of future growth rates, and low P/E stocks, or value stocks, have sunk “too low,” because investors are excessively pessimistic. If true, the subsequent high returns to value stocks and low returns to growth stocks represent simple regression toward the mean.

  Examples of regression toward the mean can be found in every aspect of life. If a basketball player scores 50 points in a game, a personal best, it is highly likely that he will score fewer points the next game. And similarly, if he scores three points, his worst game in two years, it is almost certain that he will do better the next game. Children of seven-foot-tall basketball players are tall—but not usually as tall as that. And so forth. Werner and I thought that the same process might be at work in the stock market, too. Companies that are doing well for several years in a row gather an aura implying that they are a “good company,” and will continue to grow rapidly. On the other hand, companies that have been losers for several years become tagged as “bad companies” that can’t do anything right. Think of it as a form of stereotyping at the corporate level. If this corporate stereotyping is combined with the tendency to make forecasts that are too extreme, as in the sense of humor study, you have a situation that is ripe for mean reversion. Those “bad” companies are not as bad as they look, and on average are likely to do surprisingly well in the future.

  Predicting mean reversion in the stock market would not seem to be a particularly radical hypothesis, except for one thing: the EMH says it can’t happen. The price-is-right component says that stock prices will not diverge from intrinsic value, so, by definition, can’t be “cheap.” And the no-free-lunch component says that you cannot beat the market because all information is already captured in the current price. As the past history of the stock’s returns and its P/E ratio are clearly known, they cannot predict future price changes. They are SIFs. Finding evidence of mean reversion would constitute a clear violation of the EMH. So we decided to see if we could find that evidence.

  Our study was simple. We would take all the stocks listed on the New York Stock Exchange (which, at that time, had nearly all of the largest companies) and rank their performance over some time period long enough to allow investors to get overly optimistic or pessimistic about some company, say three to five years. We would call the best performing stocks “Winners” and the worst performers “Losers.” Then we would take a group of the biggest Winners and Losers (say the most extreme thirty-five stocks) and compare their performance going forward. If markets were efficient, we should expect the two portfolios to do equally well. After all, according to the EMH, the past cannot predict the future. But if our overreaction hypothesis were correct, Losers would outperform Winners.

  Such a finding would accomplish two things. First, we would have used psychology to predict a new anomaly. Second, we would be offering support for what we called “generalized overreaction.” Unlike the Kahneman and Tversky experiment in which subjects were overreacting to measures of sense of humor when predicting GPA, we were not specifying what investors were overreacting to. We were just assuming that by driving the price of some stock up or down enough to make it one of the biggest winners or losers over a period of several years, investors were likely to be overreacting to something.

  The results strongly supported our hypothesis. We tested for overreaction in various ways, but as long as the period we looked back at to create the portfolios was long enough, say three years, then the Loser portfolio did better than the Winner portfolio. Much better. For example, in one test we used five years of performance to form the Winner and Loser portfolios and then calculated the returns of each portfolio over the following five years, compared to the overall market. Over the five-year period after we formed our portfolios, the Losers outperformed the market by about 30% while the Winners did worse than the market by about 10%.

  Not long after getting these results, we caught a lucky break. Hersh Shefrin had been asked to organize a session at the American Finance Association (AFA) annual meeting, and invited Werner and me to present our findings there. At that time the Journal of Finance, the official print outlet of the AFA, produced one issue per year that was devoted entirely to papers from the annual meeting. The way it worked is that the person organizing the session could nominate one paper from the session, and the current president of the AFA would choose some of those papers to publish. The selected papers were published just months later, and did not go through the formal process of peer review. Poor Hersh had a dilemma. Should he recommend his own paper that he would present at the conference, or ours? (The third paper in our session was not eligible because it had been submitted for publication already.) Hersh combined the wisdom of Solomon with a little old-fashioned chutzpah and nominated both papers. Here is where the luck comes in. The president of the American Finance Association that year was the late Fischer Black, the coinventor of the Black–Scholes option pricing formula. Black was a bit of a renegade, and he chose to publish both papers.

  My paper with Werner, published in 1985, has since become well known. But I am convinced that if Hersh hadn’t offered the back door entrée to the journal, it would have taken years to get the results published, or the paper might not have been published at all. First of all, everyone “knew” that our results—which were clear violations of the EMH—had to be wrong, so referees would have been highly skeptical. And there is no way we would have agreed to write an apologetic conclusion of the sort that had been foisted on Professor Basu. Werner was too principled, and I was too stubborn.

  ________________

  * There was actually a third condition I am leaving out for simplicity, in which subjects were told a student’s decile score on a test of mental concentration. The results of this condition lie between the other two.

  23

  The Reaction to Overreaction

  With the facts confirmed—that “Loser” stocks did earn higher returns than the market—there was only one way to save the no-free-lunch component of the EMH, which says it is impossible to beat the market. The solution for the market efficiency folks was to fall back on an important technicality: it is not a violation of the efficient market hypothesis if you beat the market by taking on more risk. The difficulty comes in knowing how to measure risk.

  This subtlety was first articulated by Eugene Fama. He correctly pointed out that all tests of the no-free-lunch component of market efficiency were actually “joint tests” of two hypotheses: market efficiency and some model of risk and return. For example, suppose someone found that new firms have higher returns than old firms. This would seemingly be a rejection of market efficiency; because the age of a firm is known, it cannot be used to “beat” the market. But it would not be a definitive rejection of market efficiency because one could plausibly argue that new firms are riskier than old ones, and that their higher returns are just the compensation rational investors require to bear the additional risk.

  This joint hypothesis argument applies to any apparent violation of the EMH, including those of Graham, Basu, Dreman, and others who claimed that value stocks were good investments. If our Loser portfolio was riskier than the Winner portfolio, then the observed higher rate of return could be the compensation rational investors demand to invest in risky portfolios. The central question became whether to accept our interpretation of our findings as evidence of mispricing,* which goes against t
he EMH, or to say they were attributable to risk.

  To answer that question, you need a way to measure risk. Assuredly the stocks in the Loser portfolio are individually risky, and some of those companies actually may go bankrupt. But in our study we had already accounted for this risk. If one of the stocks in either of the portfolios was delisted by the New York Stock Exchange (because of bankruptcy, for example), then our computer programs hypothetically “sold” the stock at whatever price could be obtained if it were listed on another exchange, or we recorded the investment as total loss. So the possibility of stocks going bankrupt was not the hidden source of risk that could explain our results.

  Still, those Loser stocks certainly did look risky. And might not scary-looking stocks, such as those whose prices had plummeted, have to earn a higher rate of return (a “risk premium”) in the market? You might think so, but such thinking was not kosher in modern financial economics. At that time, the right and proper way to measure the risk of a stock was to use the capital asset pricing model (CAPM) developed independently by financial economists John Lintner and William Sharpe.

  According to the CAPM, the only risk that gets rewarded in a rational world is the degree to which a stock’s return is correlated with the rest of the market. If you form a portfolio composed of a bunch of highly risky stocks whose prices bounce around a lot, the portfolio itself will not be especially risky if the price movements of each of the component stocks are independent of one another, because then the movements will on average cancel out. But if the returns on the stocks are positively correlated, meaning they tend to go up and down together, then a portfolio of volatile stocks remains pretty risky; the benefits of diversification conferred by holding a portfolio of the stocks are not as great. In this way, according to the CAPM, the correct measure of the riskiness of a stock is simply its correlation with the rest of the market, a measure that is called “beta.”† Roughly speaking, if a stock has a beta of 1.0, then its movements are proportional to the overall market. If a stock has a beta of 2.0, then when the market goes up or down by 10% the individual stock will (on average) go up or down by 20%. A stock that is completely uncorrelated with the market has a beta of zero.