Just before Christmas past, I was hoping to interview David Samborsky from Compuvision (the maker of TradeSim). But, to cut a long story short, things didn’t go to plan and we never did get to record that interview.
Instead, I emailed Dave a few questions including those submitted by my clients and he’s kindly answered them below. Check them out now and if you’ve got any further questions be sure to post them on the blog – I can then follow up with Dave and get you the answers you need.
Why is backtesting your trading system so important?
It’s important for a number of reasons:
- Backtesting should be used as a good estimator of future performance but should not be used as a guarantee of future performance.
- Backtesting can be used to stress test your system over periods of adverse market conditions in order to see how your system holds up.
- Backtesting can be used to quickly find the conditions under which your system breaks down.
- Backtesting can be used to quickly test a system that would otherwise take years to paper trade.
- Use back testing to quickly test what-if scenarios.
What are your thoughts on mechanical trading vs intuitive?
Mechanical trading takes away human emotional as well as the temptation to change the system you started with. Because humans have a tendency to always want to be right this tendency works counter intuitively to mechanical trading which is better explained by the law of averages rather than being able to predict a specific outcome with any great accuracy.
One of the most important aspects of back testing results is a smooth equity curve with a minimum drawdown especially over periods of adverse market conditions. Some people try to minimize their string of losses because this can have a dramatic impact on the trader’s emotion, psyche and stress levels.
However with a portfolio trading system a one off back test can be misleading because there are many ways to trade the system using available capital and trading opportunities. Compared to a single security system, back testing a portfolio trading system requires a different approach that makes use of advanced statistical analysis techniques that takes advantage of the variance in the system.
For a portfolio analysis a thorough statistical analysis is imperative. TradeSim has a Monte Carlo analysis feature which tests a portfolio system in much the same way you would if you got 100 or a 1000 traders to trade the same system using the same stocks and then analysing all of the results using statistical techniques.
What are the limitations of backtesting ?
The main limitation of any back testing is that no back testing can guarantee future performance so in essence it should be used as a tool to estimate future performance and test the robustness of a system rather than look for a guaranteed future performance. My recommendation is to thoroughly test your systems over periods of adverse market conditions such as corrections and crashes. Seeing how your system would hold up in those conditions is a good indication of how it will fare in the future.
The other limitation of backtesting is actually not being able to replicate exactly what would have happened if you did actually trade the system at the time. Such things as order entry and exit slippage etc cannot be duplicated exactly with back testing. However when statistically analysing a system the effect of these discrepancies can more or less be minimized.
What is the biggest lesson you’ve learn over the years with your involvement in TradeSim?
The biggest lesson with back testing is that it is a mistake to assume that a one off back test that would be applied to a single security trading system is adequate to characterise and quantify a portfolio trading system. It is not !!
Now for some questions from my clients… Click Here To Read More