augments, not replaces, traditional economics.
What have we learned from behavioral finance?
First, behavioral finance has added to our understanding of
market dynamics. Even in the presence of rational traders and
arbitrageurs, irrational “noise” traders will prevent efficiency. And
although it is possible to justify the existence of bubbles and
crashes within a rational expectations framework (for example,
40
Diba and Grossman, 1988), a behavioral approach gives more
reasonable explanations (for example, Abreu and Brunnermeier,
2003, and De Grauwe and Grimaldi, 2004).
Second, we are now aware of a number of biases, systematic
misjudgments that investors make. Examples include the
following:
Overconfidence: Overconfidence is an unreasonable belief
in one's abilities. This leads traders to assign too narrow a
range of possibilities to the outcome of an event, to
underestimate the chances of being wrong, to trade too large,
and to be too slow to adapt.
Overoptimism: Overconfidence compresses the range of
predictions. Overoptimism biases the range, so traders
consistently predict more and better opportunities than really
exist.
Availability heuristic: We base our decisions on the most
memorable data even if it is atypical. This is one reason teeny
options are overpriced. It is easy to remember the dramatic
events that caused them to pay off, but hard to remember the
times when nothing happened and they expired worthless.
Short-term thinking: This thinking shows the irrational
preference for short-term gains at the expense of long-term
performance.
Loss aversion: Investors dislike losses more than they like
gains. This means they hold losing positions, hoping for a
rebound even when their forecast has been proven wrong.
Conservatism: Conservatism is being too slow to update
forecasts to reflect new information.
Self-attribution bias: This bias results from attributing
success to skill and failure to luck. This makes Bayesian
updating of knowledge impossible.
Anchoring: Anchoring occurs when relying too much on an
initial piece of information (the “anchor”) when making a
forecast. This leads traders to update price forecasts too slowly
because the current price is the anchor and seems more
“correct” than it should.
And there are at least 50 others.
41
It is these types of biases that traders have tried to use to find trades with edge. Results have been mixed. There are so many
biases that practically anything can be explained by one of them.
And sometimes there are biases that are in direct conflict. For
example, investors underreact, but they also overreact. Between
these two biases you should be able to explain almost any market
phenomena. The psychologists and finance theorists working in
the field are not stupid. They are aware of these types of
difficulties and are working to disentangle the various effects. The
field is a relatively new one and it is unfair and unrealistic to
expect there to be no unresolved issues. The problem is not really
with the field or the serious academic papers. The problem is with
pop psychology interpretations and investors doing “bias mining”
to justify ideas.
It is common in science for a new idea to be overly hyped,
particularly those that are interesting to lay people (traditional
finance is not interesting). In the 1970s there were popular books
about catastrophe theory, a branch of physics that was meant to
explain all abrupt state changes and phase transitions. It didn't. In
the 1990s, chaos theory was meant to explain practically
everything, including market dynamics. It didn't. Behavioral
finance is being overexposed because it is interesting. It provides
plenty of counterintuitive stories and also a large amount of
schadenfreude. We can either feel superior to others making
stupid mistakes or at least feel glad that we aren't the only ones
who make these errors.
And people love intuitive explanations. We have a great need to
understand things, and behavioral finance gives far neater
answers than statistics of classical finance theory. Even though
behavioral finance doesn't yet have a coherent theory of markets,
the individual stories give some insight. They help to demystify.
This is reassuring. It gives us a sense of control over our
investments.
A science becoming interesting to the general public doesn't
necessarily mean it is flawed. For example, there have been
hundreds of popular books on quantum mechanics. However,
behavioral finance does have some fairly serious problems to
address.
Just as in conventional finance theory, behavioral finance studies
individual decision-making despite the fact that people do not
42
make investing decisions independently of the rest of society.
Everyone is influenced by outside factors. Most people choose
investments based on the recommendations of friends (Katona,
1975). And professionals are also influenced by social forces
(Beunza and Stark, 2012). Over the last 30 years the sociology of
markets has been an active research field (for example, Katona,
1975, Fligstein and Dauter, 2007, and references therein), but this
work hasn't yet been integrated into behavioral finance. Because
behavioral finance largely ignores the social aspects of trading and
investing, we don't have any idea of how the individual biases
aggregate and their net effect on market dynamics. This is
necessary because, even though we don't understand how
aggregate behavior emerges, it is very clear that markets cater to
irrational behavior rather than eradicate it. For example, the
services of financial advisors, stock brokers, and other financial
intermediaries made up 9% of the US GDP (Philippon, 2012)
despite the fact that they are almost all outperformed by much
cheaper index funds and ETFs.
Next, behavioral finance has largely limited itself to the study of
cognitive errors. There are many other types of nonrational
behavioral inputs into decision-making, including emotion,
testosterone levels, substance abuse, and the quest for status.
And behavioral finance gives no coherent alternative theory to the
EMH. A catalog of biases and heuristics—the mistakes people
make—is not a theory. A list of facts does not make a theory. Of
course, sometimes observations are necessary before a theory can
be formulated. Mendeleev drew the periodic table well before the
atomic structure of matter was understood. We knew species
existed well before we understood the process of speciation by
natural selection. Still, to be scientific, behavioral finance
eventually needs to lead to a unifying theory that gives
explanations of the current observations and makes testable
predictions.
Behavioral finance can still help. Whenever we find somethi
ng
that looks like a good trading idea we need to ask, “Why is this
trade available to me?” Sometimes the answer is obvious. Market-
makers get a first look in exchange for providing liquidity. Latency
arbitrage is available to those who make the necessary
investments in technology. ETF arbitrage is available to those with
the capital and legal status to become authorized participants. But
43
often a trade with positive edge is available to anyone who is
interested. Remembering the joke about the economists, “Why is
this money sitting on the ground?” Risk premia can often be
identified by looking at historical data, but behavioral finance can
help to identify real inefficiencies. For example, post-earnings
announcement drift can be explained in terms of investor
underreaction. Together with historical data, this gives me enough
confidence to believe that the edge is real. The data suggest the
trade, but the psychological reason gives a theoretical justification.
High-Level Approaches: Technical Analysis
and Fundamental Analysis
Technical analysis is the study of price and volume to predict
returns.
Technical Analysis
Aronson (2007) categorized technical analysis as either subjective
or objective. It is a useful distinction.
Subjective technical analysis incorporates the trader's discretion
and interpretation of the data. For example, “If the price is over
the EWMA, I might get long. It depends on a lot of other things.”
These methods aren't wrong. They aren't even methods.
Subjectivity isn't necessarily a problem in science. A researcher
subjectively chooses what to study and then subjectively chooses
the methods that make sense. But if subjectivity is applied as part
of the trading approach, rather than the research, then there is no
way to test what works and what doesn't. Do some traders succeed
with subjective methods? Obviously. But until we also know how
many fail, we can't tell if the approach works. Further, the
decisions different traders who use ostensibly the same method
make won't be the same or even based on the same inputs. There
is literally no way to test subjective analysis.
Some things that are intrinsically subjective are Japanese
candlesticks, Elliot waves, Gann angles, trend lines, and patterns
(flags, pennant, head, and shoulders, etc.). These aren't methods.
In the most charitable interpretation, they are a framework for
(literally) looking at the market. It is possible that using these
methods can help the trader implicitly learn to predict the market.
But more realistically, subjective technical analysis is almost
44
certainly garbage. I can't prove the ideas don't work. No-one can.
They are unfalsifiable because they aren't clearly defined. But
plenty of circumstantial evidence exists that this analysis is
worthless. None of the large trading firms or banks has desks
devoted to this stuff. They have operations based on stat arb, risk
arb, market-making, spreading, yield curve trading, and volatility.
No reputable, large firm has a Japanese candlestick group.
As an ex-boss of mine once said, “That isn't analysis. That is
guessing.”
Any method can be applied subjectively, but only some can be
applied objectively. Aronson (2007) defines objective technical
analysis as “well-defined repeatable procedures that issue
unambiguous signals.” These signals can then be tested against
historical data and have their efficacy measured. This is essentially
quantitative analysis.
It seems likely that some of these approaches can be used to make
money in stocks and futures. But each individual signal will be
very weak and to make any consistent money various signals will
need to be combined. This is the basis of statistical arbitrage. This
is not within the scope of this book.
However, we do need to be aware of a bad classic mistake when
doing quantitative analysis of price or return data: data mining.
This is where we sift through data using many methods,
parameters, and timescales. This is almost certain to lead to some
strategy that has in-sample profitability. When this issue is
confined to choosing the parameters of a single, given strategy it is
usually called overfitting. If you add enough variables, you can get
a polynomial to fit data arbitrarily well. Even if you choose a
function or strategy in advance, by “optimizing” the variables you
will the get the best in-sample fit. It is unlikely to be the best out of
sample. Enrico Fermini shared that the mathematician and
economist John von Neumann said, “With four parameters I can
fit an elephant , and with five I can make him wiggle his trunk”
(Dyson, 2004).
This mistake isn't only made by traders. Academics also fall into
the trap. The first published report of this was Ioannidis (2005).
Subsequently, Harvey et al. (2016) and Hou et al. (2017) discussed
the impact of data mining on the study of financial anomalies.
There are a few ways to avoid this trap:
45
The best performer out of a sample of back-tested rules will be
positively biased. Even if the underlying premise is correct, the
future performance of the rule will be worse than the in-
sample results.
The size of this bias decreases with larger in-sample data sets.
The larger the number of rules (including parameters), the
higher the bias.
Test the best rule on out-of-sample data. This gives a better
idea of its true performance.
The ideal situation is when there is a large data set and few
tested rules.
Even after applying these rules, it is prudent to apply a bias
correcting method.
The simplest is Bonferroni's correction. This scales any statistical
significance number by dividing by the number of rules tested. So,
if your test for significance at the 95% confidence level (5%
rejection) shows the best rule is significant, but the rule is the best
performer of 100 rules, the adjusted rejection level would be
5%/100 or 0.005%. So, in this case, a t-score of 2 for the best rule
doesn't indicate a 95% confidence level. We would need a t-score
of 2.916, corresponding to a 99.5% level for the single rule. This
test is simple but not powerful. It will be overly conservative and
skeptical of good rules. When used for developing trading
strategies this is a strength.
A more advanced test is White's reality check (WRC). This is a
bootstrapping method that produces the appropriate sampling
distribution for testing the significance of the best strategy. The
test has been patented and commercial software packages that
implement the test can be bought. However, the basic algorithm
can be illustrated with a simple example.
We have two strategies, A and B, which produce daily returns of
2% and 1% respectively. Each was developed by looking at 100
historical returns. We can use WRC to determine
if the apparent
outperformance of strategy A is due to data mining:
Using sampling with replacement, generate a series of 100
returns from the historical data.
46
Apply the strategies (A and B) to this ahistorical data to get the
pseudo-strategies A' and B'.
Subtract the mean return of A from A' and B from B'.
Calculate the average return of the return-adjusted strategies,
A” and B”.
The larger of the returns of A” and B” is the first data point of
our sample distribution.
Repeat the process N times to generate a complete
distribution. This is the sampling distribution of the statistic,
maximum average return of the two rules with an expected
return of zero.
The p-value (probability of our best rule being truly the better
of the two) is the proportion of the sampling distribution
whose values exceed the returns of A, that is, 2%.
A realistic situation would involve comparing many rules. It is
probably worth paying for the software.
There is also a totally different and complementary way to avoid
overfitting. Forget about the time series of the data and study the
underlying phenomenon. A hunter doesn't much care about the
biochemistry of a duck, but she will know a lot about their actual
behavior. In this regard a trader is a hunter, rather than a
scientist. Forget about whether volatility follows a GARCH(1,1) or
a T-GARCH(1,2) process; the important observation is that it
clusters in the short term and mean reverts in the long term. If the
phenomenon is strong enough to trade, it shouldn't be crucial
what exact model is used. Some will always be better in a sample,
but that is no guarantee that they will work best out of a sample.
As an example, this is the correct way to find a trading strategy.
There is overwhelming evidence that stocks have momentum.
Stocks that have outperformed tend to continue outperforming.
This has been observed for as long as we have data (see Geczy and
Samnov [2016], Lempérière et al. [2014], and Chabot et al.
[2009]) and in many countries (for example, Fama and French,
2010). The observation is robust with respect to how momentum
is defined and the time scales over which it is measured. In the
trading world, the evidence for stock momentum is overwhelming.
Positional Option Trading (Wiley Trading) Page 5