5 Things You Learn From Backtesting Trading Strategies

When people talk about backtesting their minds often turn to formulas and equity curves. Some may even be drawn to data mining or curve fitting, things that can go wrong.

But backtesting is not just a way to test the profitability of a trading signal. In fact, backtesting has taught me many important lessons about trading and investing in general.

I thought I would share some of those lessons below.

1. The Influence Of Randomness

One of the most important things backtesting has taught me over the years is the influence of randomness on financial markets. 

I’ve repeatedly found that random trading strategies (given appropriate holding periods) can hold their own amongst other classic strategies and even outperform them. 

This shows the amount of randomness in financial data and the efficiency that exists in most markets.

Once you realise that a completely random trading strategy has the ability to outperform buy-and-hold, you tend to view investing in a new light. 

Also, backtesting allows you to test strategies against completely random models and therefore see whether they are any better than random.

You can also introduce random elements into a trading strategy in order to test robustness. 

For example, random exit signals, holding lengths, indicator parameters or ranking scores. Using randomness in this way can be used to stress test your strategy and make sure it is robust across different dimensions.

2. The Size Of An Edge

Sometimes you have an idea for a trading strategy that seems to make sense and works well when applied to a couple of different charts.

However, without properly backtesting that strategy, it is impossible to know the key metrics that influence how you should trade it or allocate capital. 

By backtesting a strategy you can see the size of the edge, the win rate, the drawdown and the expectancy.

This information is crucial because it allows you to see if your profit is going to be big enough to overcome trading costs (like slippage). 

It also allows you to calculate losing streaks so you can decide whether you have the psychological fortitude to follow a system through bad periods.

Most importantly, this information shows you the historical payoff and therefore tells you whether or not you should allocate capital to the strategy and how much. 

Going a step further, once you have this information to hand, you can use backtesting to try out different money management techniques such as Kelly, fixed risk, Williams etc.

3. Answers To Crucial Questions

A classic example of how backtesting is useful is that it lets you see the historical relationships of things. 

You might have been told, for example, that bonds have a negative correlation to stocks, or that gold is a hedge against inflation. 

But how can you trust these words without going back in time and seeing if they’re true? 

With backtesting, you can look at the past and see how certain correlations hold up. And you can see how those relationships react under different market conditions.

More importantly, backtesting allows you to answer critical portfolio questions like: 

• What is the optimal number of stocks to hold in a portfolio?
• Is there an optimal rebalancing period?
• What is the optimal allocation mix?
• How many strategies or sectors to include in a portfolio?

Without backtesting, answering these questions is like throwing darts in the dark.

4. Skepticism And Simplicity

When you have been backtesting for a while you start to understand that many of the strategies people talk about are not as good as they seem. 

Backtesting gives you the power to test strategies and see if they hold up to scientific analysis.

It therefore builds up a healthy level of skepticism which actually puts you at a decreased risk of fraud (or at least not being fooled by a fake guru). 

With time, backtesting reveals the importance of simplicity too. 

You backtest hundreds of strategies only to find many that are overly complex and blow up in real trading. 

Eventually you realise that you can make more money by sticking to a simple strategy but giving it more of your focus and capital. 

5. Backtesting Breeds Confidence In Simple Frameworks

For me, the most valuable thing about backtesting is that it gives me the ability to follow simple investment frameworks with greater confidence.

For example, a simple momentum or dollar cost averaging strategy, can be shown to be, at a high level, robust enough to follow with conviction.

Across thousands of backtests I can learn what is a good number of stocks to hold in a portfolio and how long to hold them for. These are critical issues that an investor needs to know.

When markets are plummeting, it is all too easy to abandon a strategy out of panic and join the herds of investors running for the exit.

But backtests remind us that the most important thing during times of panic is usually to stay the course and not to jump ship at the worst time.

Confidence could be the most important quality for any investor and backtesting provides it.

The Truth About Backtesting

At the end of the day, backtesting is not perfect and of course there will be problems and issues to be aware of such as curve fitting data and over optimisation.

But when applied correctly, backtesting is a powerful way to answer questions and analyse the robustness of a particular investment style.

When someone tells you that value investing, or momentum, or buy-and-hold is the best strategy there is no way of knowing if they are right or not. But backtesting allows you to put all those strategies to the test.

Essentially, backtesting can be used to tell you the truth. It is therefore an invaluable tool that all investors should utilise.

---