How to Use the Past to Predict the Future

Asad Dossani
Any algorithmic trading strategy must meet two criteria. The first is that the set of rules we create must completely specify our trading strategy. The second is that we must use historical data to create the algorithm. For a full discussion of these criteria, please refer to our previous newsletter.

Let's explore this second criteria in more detail. Why do we use historical data? The historical data is used to test our algorithmic strategies. We test our strategies on historical data to see how it would have performed had we implemented it in the past. This is called backtesting.

If we want to trade an algorithm live, we need to ensure that it performed well on historical data. This gives us an idea of what kind of performance to expect from our algorithm in the future. The process of backtesting allows us to try different algorithms and only select the best ones.

What is the implicit assumption we are making anytime we use backtesting to select trading strategies? We are assuming that if an algorithm performed well in the past, it will also perform well in the future. Another way to put it is that the market patterns we observe in the past will repeat themselves in the future.

This may seem a like a strong assumption that is unlikely to hold in practice. The truth is that market conditions do change over time. The market patterns we observe in the past will not perfectly replicate themselves in the future.

However, just because market conditions change doesn't mean that past patterns won't repeat themselves in the future. While market patterns do change, they usually change slowly. This means that over short intervals, market conditions are going to be similar. As intervals get longer, market conditions will look increasingly different.

When we backtest a trading strategy, the best indicator of future performance is how the strategy performed on the most recent historical data. Suppose we backtest a strategy using historical data in 2013. Then it is likely that the 2014 performance will be similar to the 2013 performance. If we backtest a strategy using historical data from 2010, then this will be less similar to the 2014 performance.

This is the main reason that backtesting works. Even though the future is not exactly the same as the past, it may be close enough that our trading strategies will be profitable in a live market. Still, we should expect that there will be some difference between the backtest result and the future performance of a strategy.

There is one exception to this. Sometimes the market condition can change dramatically in a short space of time. This often occurs during financial crises. In this case, a trading algorithm may stop working right away. Unfortunately, it is impossible to predict in advance when this will occur. This is one of the risks of algorithmic trading, and is in fact a risk of any trading strategy.

Let's go back to our assumption. Why should we rely on what happened in the past if we know that the future will be different? We do this because it is the best we can hope to do. In fact, this assumption is present for any type of investment style.

Take value investing as an example. In the past, we have observed that value stocks generate the highest returns in the long run. Due to this past behavior of value stocks, we buy value stocks today expecting that the same thing will occur. What we are doing is assuming that the future will be similar to the past.

This applies not just to trading or investing, but any choice we make. All our experiences of the past contribute to our decision making. We make choices about the future based on what we've observed in the past.

Back to algorithmic trading. Maybe it should no longer seem surprising that we assume the future will be like the past. Even though it is a strong assumption that doesn't hold exactly in practice, it's still close enough. The future is never exactly the same as the past, but it is still similar.

We need to be aware of this assumption whenever we test algorithmic trading strategies. I have often found that the performance of an algorithm during periods of financial crises is different to its performance in more normal market conditions.

The trick to a successful strategy is to find one that makes money in at least one market condition, and doesn't lose money in the other. So if my strategy makes money during normal market conditions, and remains flat during crises, then that is a winning strategy.

Get Asad Dossani's Best Short Term Investment
Opportunities Delivered Straight To Your Inbox!

Sign Up For Profit Hunter Today... It's Free!
We hate spam as much as you do. Check out our Privacy Policy and Terms Of Use
We request your view! Post a comment on "How to Use the Past to Predict the Future". Click here!
23 Responses to "How to Use the Past to Predict the Future"
04 May, 2017
22 Oct, 2014
give me detailsLike 
06 Jul, 2014
Mr. A. Dossain, It is sad to inform U that, in our busy schedules of life, we have hardly any time to listen and read your explanatory messages in financial arena. Moreover, as on date, at best I might have gone through 10% of your messages, not even read your article. So pl cut short your articles and finally come on to the point instead of more and more highly explanatory notes. Sir, sorry and I think U know better a lay man expectation from stock market THANKS, SUBAS MASANTA 6.7.14 11.15 pmLike (2)
06 Jul, 2014
Dear Assad, it will be of great help if u show the replies for the queries of all investors, so that we can clarify all our doubts and there will be no need to repeat the same doubts again and again. where can we see the past performance record ? pls explain with examples using some stock for clear understanding.Like 
Ravi N Patel
06 Jul, 2014
I want to read this types of News and articles daily basis so i request you to send me daily. Thanks. Like 
06 Jul, 2014
Dear Asad, the concept you have espoused is extremely compelling and has kicked up tremendous interest. However is it an investment strategy based on fundamentals or a trading idea to profit from, for the short term? Like 
03 Jul, 2014
how too use past to prdict future;is commansense idea;how to get past record is difficult for new scrip.what to do for that? algorithmic trading is god idea , but cutomer hod =pick the sctips hose last fue days is rising and that he note dowans.that is calld algorithamic calculation.But for more past record what to do ?who supply past record of certain my selected scripts?Like 
27 Jun, 2014
I wish to know more and wish to subsribeLike 
26 Jun, 2014
Interesting and simple to understand. Algorithm - is a set of steps to solve a mathematical problem - is based on past data. 'History repeats itself' is a well known saying. Thanks Asad for this lucid letter.Like 
Ali Farhat
26 Jun, 2014
I how can be useful memberLike 
26 Jun, 2014
Dear Assad you are building a good base for curiosity among readers to look forward your next articles. Yes you are right past can be a guiding factor for basing future decision in a totally uncertain times .useful article for new investors. Like 
26 Jun, 2014
The trick to a successful strategy is to find one that makes money in at least one market condition, and doesn't lose money in the other.Like 
kothwal venkata ramana
26 Jun, 2014
past experience is useful to understand market situationLike (1)
25 Jun, 2014
Stress testing an algo is as important as back testing. The desired result may be closure of all open positions, but even this must be tested.Like 
25 Jun, 2014
Very good logic. I largely agree with this.Like 
niilesh Zaveri
25 Jun, 2014
Dear Assad, The explanation of concept & logic is ok, but need to know how to use it practically with live examples from Indian Stock market. Also need to know from where to gather the info reqd for completing back testing for Algorythmic trades. are there ready made softwares available which has this kind of parameters captured on real time basis. If yes, the cost of using the same.Like (1)
Digambar Kulkarni
24 Jun, 2014
Market sentiment is based on Human Behavior. Human beings are quick in learning, however, as you predict, mass learning will be delayed. During that period, old behavior pattern is sure to repeat, other factors remaining constant. I sincerely feel, this approach will be very useful!Like 
24 Jun, 2014
Dear sir, I have been following your posts with interest. I like your step by step logical approach. I would appreciate it if you could walk us through algorithmic investment without complex mathematics and jargon so that an average retail investor could benefit. Thank you. Like 
24 Jun, 2014
You use the term "trading" often.Is this day trading or investment for a longer period?Like 
24 Jun, 2014
Dear Asad, Thank you for the articles describing the fundamentals. I can see that you are slowly building up the audience knowledge to the levels that is required to understand algorithmic trading strategies and how it works. Awaiting for your next article. Dear Fellow readers, please do not jump the gun. Lets learn the fundamentals and then the specifics of algorithmic trading strategies Mr Asad is trying to teach us. Lets ask doubts related to the article(s) - and not - how and why we can make/predict profit/market. I am sure Mr Asad will let us know how the complex algorithmic strategies can be made/maintained and tested/put-into-practice for us to make money in his future articles. Lets give feedbacks that will improve the future articles - so that all of us benefit from this series. Happy learning! And once again, thanks to Mr Asad and EquityMaster for bringing this advanced topic to our, i.e., non-algorithmic traders', understanding. best regards, S.T.Like 
madhav ranade
24 Jun, 2014
i asked a question after last article which has not been replied. I believe yr articles are for retail traders / investors. pl let me know which broker is allowing algo trading facility to retail clients. if NO ONE is offering, why r we discussing this ? I hope now atleast someone from EM will respond.Like 
24 Jun, 2014
a very interesting beginning,especially for those mathematically challenged who think that ALGRITHM is something incomprehensive. waiting anxiously for your next step .Asad.Like 
Rajesh Cherayil
24 Jun, 2014
" History may not repeat itself but it does rhyme." Mark TwainLike 
We request your view! Post a comment on "How to Use the Past to Predict the Future". Click here!