Note that using a moving average as a variable of interest reduces the size of your data set, since the initial values will not have a moving average calculated. You need to take this into account when determining your desired sample size.
We now double sort the data to perform the contextual investigation. Let’s say that we’re interested in the expectations following a rising day in a market that is trading above versus below its 20-day moving average. We sort the data based on column F “SP(1)” as we did in our previous lesson, performing the sort on a Descending basis, so that the largest and positive price changes appear at the top of the sheet. Now we select only the data for the cells that show positive price change and copy those to another sheet. Our second sort will be based on column G (MA), again on a Descending Basis. This will separate the up days in markets trading above their 20-day moving average (those coded “1”) from all other days.
As before, we’ll examine the average next day’s return by calculating the average for the cells in column H that are coded in column G as “1” and comparing that to the average for the cells in column H that are coded in column G as “0.”
Thus, let’s say that there are 538 cells for up days; 383 of these are coded “1” in column G and 155 are coded “0.” You would compare “=average(H2:H384)” and “=average(H385:H539).” You could also code the cells in column H as either “1” or “0” in column I based on whether they are up or down “=if(H2>0,1,0)” and then compare “=sum(I2:I384)” and “=sum(I385:I539)” to see if there are notable differences in the number of up days following up days in markets that are above and below their moving averages.
Just for your curiosity, using cash S&P 500 data as the raw data, I found that the average next day change following an up day when we’re above the 20-day moving average to be -0.04 percent; the average next day change following an up day when we’re below the 20-day moving average was -0.18 percent. This is a good example of a finding that doesn’t knock my socks off, but is suggestive. I would want to conduct other investigations of what happens after rising days in falling markets before generating trading hypotheses that would have me shorting strength in a broader downtrend.
Many times you’ll see differences in the sorted data that are strong enough to warrant further investigation, but not strong enough to justify a trading hypothesis by itself.
This combination of coding and sorting can create a variety of contextual views of markets. For example, if we type in, “=if(E21=max(E2:E21), 1,0)” we can examine the context in which the current day is the highest price in the past 20 and see how that influences returns. If we include a second independent variable, such as the number of stocks making new 52-week highs and lows, we can examine how markets behave when new highs exceed new lows versus when new lows exceed new highs. For instance, if new highs go into column F and new lows into column G, we can code for “=if(F21>G21,1,0” in column H, place our dependent measure (perhaps the next day’s price change) in column I and sort based on the new high/low coding.
As mentioned earlier, it is wise to not create too many contextual conditions, because you will wind up with a very small sample of occasions that fit your query, and generalization will be difficult. If you obtain fewer than 20 occasions that meet your criteria, you may need to relax those criteria or include fewer of them.
As your own trading coach, you can utilize these contextual queries to see how markets behave under a variety of conditions. The movement of sectors, related asset classes—anything can be a context that affects recent market behavior. In exploring these patterns, you become more sensitive to them in real-time, aiding your selection and execution of trades.
COACHING CUE
If you’re interested in longer-term trading or investing, you can create spreadsheets with weekly or monthly data and investigate independent variables such as monthly returns on the next month’s returns; VIX levels on the next month’s volatility; sentiment data on the next month’s returns; price changes in oil on the next month’s returns, etc. You can also code data for months of the year (or beginning/end of the month) to investigate calendar effects on returns.
LESSON 99: FILTER DATA
Let’s say you want to analyze intraday information for the S&P 500 Index futures. Now your spreadsheet will look different as you download data from sources such as your real-time charting application. Your first column will be date, your second column will be time of day, and your next columns will be open, high, low, and closing prices. If you so select, the next column can be trading volume for that time period (one-minute, five-minute, hourly, and so on).
Suppose you want to see how the S&P 500 market has behaved at a certain time of day. What we will need to do is filter out that time of day from the mass of downloaded data and only examine that subset. Instead of sorting data, which has been a mainstay of our investigations to this point, we will use Excel’s filter function.
To illustrate how we might do this, we’ll start with a simple question. Suppose we want to know how trading volume for the current first half-hour of trading compares with the average trading volume for that corresponding half-hour over the prior 20 days of trading. This will give us a rough sense of market activity, which correlates positively with price volatility. The volume also gives a relative sense for the participation of large, institutional traders. If, say, we observe a break out of a range during the first 30 minutes of trading, it is helpful to know whether or not these large market-moving participants are on board.
Volume analyses can help you identify who is in the market.
For this investigation, we’ll examine half-hourly data for the S&P 500 emini contract. I obtain my intraday data from my quote platforms; in the current example, I’ll use e-Signal. To do this, we create a 30-minute chart of the ES futures contract; click on the chart and scroll to the right to move the chart backward in time. When we’ve covered the last 20 days or so, we click on the menu item Tools; select Data Export; then uncheck the boxes for the data that we won’t need. In this case, all we’ll need is Date, Time, and Volume. We click the button for Copy to Clipboard and open a fresh sheet in Excel. Once we click on the Excel menu item for Edit and select Paste, with the cursor at cell A2, we’ll populate the sheet with the intraday data. We can then enter names for the columns in row 1: Date; Time; and Volume. (If you’re downloading from e-Signal, those names will accompany the data and you can download the data with the cursor at A1).
Our next step is to highlight the entire data set that we want to cover. We click on the Excel menu item for Data; select Filter; and select AutoFilter. A set of small arrows will appear beside the column names. Click the arrow next to Time and, from the drop down menu, select the time that represents the start of the trading day. In my case, living in the Chicago area in Central Time, that would be 8:30 A.M. You’ll then see all the volume figures for the half-hour 8:30 A.M. to 9:00 A.M. Click on Edit; select Copy; open a blank sheet; click on Edit; and select Paste. This will put the 8:30 A.M. data on a separate sheet. If you have 20 values (the past 20 days), you can enter the formula “=average(c2:c21)” and you’ll see the average trading volume for the first half-hour of trading. Of course, you can filter for any time of day and see that half-hour’s average volume as well.
When you know the average trading volume for a particular time period, you can assess institutional participation in real time—particularly with respect to whether this volume picks up or slows down as a function of market direction.
The filter function is helpful when you want to pull out data selectively from a data set. Let’s say, for instance, that you had a column in which you coded Mondays as 1; Tuesday’s as 2; etc. You could then filter out the 1s in the historical data set and see how the market behaved specifically on Mondays. Similarly, you could code the first or last days of the month and filter the data to observe the returns associated with those.
In general, I find filtering most helpful for intraday analyses, when I want to see how
markets behave at a particular time of day under particular conditions. Frankly, however, this is not where I find the greatest edges typically, and it’s not where I’d recommend that a beginner start with historical investigations. Should you become serious about investigating such intraday patterns, I strongly recommend obtaining a clean database from a vendor such as Tick Data. You can use their data management software to create data points at any periodicity and download these easily to Excel. Serious, longer-term investigations of historical intraday data need tools far stronger than Excel. Limits to the size of spreadsheets and the ease of maneuvering them make it impossible to use Excel for long-term investigations of high frequency data.
Still, when you want to see how markets behave in the short run—say, in the first hour of trading after a large gap open—investigations with intraday data and filtering can be quite useful. You’ll find interesting patterns of continuation and reversal to set up day-trading ideas or to help with the execution of longer timeframe trades.
COACHING CUE
Filtering can be useful for examining patterns of returns as a function of time of day. For instance, say the market is down over the past two hours: how do returns compare if those two hours are the first versus the last two hours of the day? How are returns over the next few hours impacted if the day prior to those two hours was down? Such analyses can be very helpful for intraday traders, particularly when you combine price change independent variables with such intraday predictors as NYSE TICK.
LESSON 100: MAKE USE OF YOUR FINDINGS
This chapter has provided only a sampling of the kinds of ways that you can use simple spreadsheets and formulas to investigate possible patterns in historical data. Remember: these are qualitative looks at the data; they are designed to generate hypotheses, not prove them. Manipulating data and looking at them from various angles is a skill just like executing trades. With practice and experience, you can get to the point where you investigate quite a few patterns all in the hour or two after market close or before they open.
The key is to identify what makes the current market unique or distinctive . Are we well below or above a moving average? Have there been many more new lows than highs or the reverse? Has one sector been unusually strong or weak? Have the previous days been strong or weak? It is often at the extremes—when indicators or patterns are at their most unusual—that we find the greatest potential edges. But sometimes those unique elements are hard to find. Very high or low volume; strong or weak put/call ratios; large opening gaps—all are good areas for investigations.
We find the greatest directional edges following extreme market events.
Once you have identified a pattern that stands out, this becomes a hypothesis that you entertain to start a trading day or week. If, say, I find that 40 of the last 50 occasions in which the market has been very weak with a high put/call ratio have shown higher prices 20 days later, this will have me looking for a near-term bottoming process. If, after that analysis, I notice that we’re making lower price lows but with fewer stocks and sectors participating in the weakness, this may add a measure of weight to my hypothesis. Eventually, I might get to the point where I think we’ve put in a price bottom and I’ll buy the market, giving myself a favorable risk/reward should the historical pattern play out.
But equally important, consider the scenario in which we see good historical odds of bouncing over a 20-day period, leading us to search for a near-term bottoming process. My fresh data, however, suggest that the market is weakening further: more stocks and sectors are making lows, not fewer. The historical pattern does not appear to be playing itself out. This, too, is very useful data. When markets buck their historical tendencies, something special may be at work. Some very good trades can proceed from the recognition that markets are not behaving normally.
This is the value of considering patterns as hypotheses and keeping your mind open to those hypotheses being supported or not. A historical pattern in markets is a kind of script for the market to follow; your job is to determine whether or not it’s following that script.
Our analyses only inform us of historical tendencies. If a market is not behaving in a manner that is consistent with its history, this alerts us to unique, situational forces at work.
All of this suggests that historical investigations are useful logical aids, but my experience is that their greatest value may be psychological. Day after day, week after week, and year after year of investigating patterns and running market results through Excel have given me a unique feel for patterns. It also has given me a keen sense for when patterns are changing: when historical precedents may no longer hold.
One routine that has been very helpful has been to isolate the last five or so instances of a potential pattern. If the market has behaved quite differently in the last several instances than it has historically, I entertain the possibility that we’re seeing a shift in market patterns. If I see the last few instances behaving abnormally across many different variables and time frames, those anomalies strengthen my sense of a market shift.
When I see how results have played out over the years, I become a less näïve trend follower. I don’t automatically assume that rising markets will continue to skyrocket or that falling markets will continue to plunge. I’ve developed tools for determining when trending markets are gaining and losing steam; these have been helpful in anticipating reversals. Seeing how these indicators behave under various market conditions over time—and actually quantifying their track record—has provided me with a measure of confidence in the ideas that I would not have in the absence of intimacy with the data.
Much of the edge in trading comes from seeing markets in unique ways, catching moves before they occur or early in their appearance. It is easy to become fixed in our views, with vision narrowed by looking at too few markets and patterns. As your own trading coach, you need to keep your mind open and fresh. Read, talk with experienced traders, follow a range of markets closely, test patterns historically, and know what’s happening globally: you’ll see things that never register on the radar of the average trader. You’ll be at your most creative when you have the broadest vision.
COACHING CUE
When you examine historical patterns, go into your data set and specifically examine the returns from the occasions that didn’t fit into the pattern. This will give you an idea of the kind of drawdowns you could expect if you were to trade the pattern mechanically. Many times, the exceptions to patterns end up being large moves; for instance, most occasions may show a countertrend tendency with a relative handful of very large trending moves. If you know this, you can look for those possible exceptions, study them, and maybe even identify and profit from them.
RESOURCES
The Become Your Own Trading Coach blog is the primary supplemental resource for this book. You can find links and additional posts on the topic of coaching processes at the home page on the blog for Chapter 10: http://becomeyourowntradingcoach.blogspot.com/2008/08/daily-trading-coach-chapter-ten-links.html
My own interest in historical patterns owes a great deal to the work of Victor Niederhoffer. His Daily Speculations web site is a source of many testable ideas regarding market movements: www.dailyspeculations.com
Henry Carstens’ online resource, An Introduction to Testing Trading Ideas, is a worthwhile and popular resource: www.verticalsolutions.com/books.html
Mike Bryant’s trading systems work is quite good; here’s a collection of free downloads from his site: www.breakoutfutures.com/PreDownload.htm
Rob Hanna’s blog tests a number of historical trading patterns and is a great stimulus for your own research: www.quantifiableedges.blogspot.com
Two subscription services that do a fine job of testing trading ideas are the SentimenTrader site from Jason Goepfert (www.sentimentrader.com) and the Market Tells letter from Rennie Yang (www.markettells.com).
Henry, Rob, Jason, and Rennie all contributed segments to Chapter 9 of this book, offering insights i
nto the relevance of testing ideas for self-coaching.
Conclusion
Science strives to achieve unity of fact. Art strives to achieve unity of feeling.
—Stephen Pepper
This is our final lesson; let’s see if we can achieve a bit of unity of feeling as well as fact.
LESSON 101: FIND YOUR PATH
My mother, Constance Steenbarger, passed away last year. She was an artist and an art teacher. Her greatest work of art, however, was her family. She provided her children—and her husband—with the one, irreplaceable psychological gift: the knowledge and the feeling that they were special. It’s amazing how much you can achieve when you know that you’re not ordinary. Out of that awareness, you’re unwilling to settle for average in your work, your relationships, or your returns from markets. When you create a work of art out of your family, you empower human beings to want to make works of art of their lives. What greater accomplishment could there be? If I can achieve, as a psychologist and parent, a bit of what Connie Steenbarger accomplished with her family, it will count more than any degree after my name, any great trade I might place.
The Daily Trading Coach Page 45