Conquering Curve Fitting: A Game-Changing Technique for Robust Systematic Trading
An Effective Technique To Build Robustness Into A Trading System
Welcome to the “Systematic Trading with TradeQuantiX” newsletter, your go-to resource for all things systematic trading. This publication will equip you with a complete toolkit to support your systematic trading journey, sent straight to your inbox. Remember, it’s more than just another newsletter; it’s everything you need to be a successful systematic trader.
Introduction:
One of the biggest and most annoying problems systematic traders face is developing robust trading systems. Have you ever developed a new trading system that looked good in the backtest but fell apart during robustness testing?
This is generally a sign of unintentional curve fitting. We don’t try to curve fit, but it’s a lot easier to do accidentally than I wish it was.
So, how can we reduce the risks of curve fitting our trading systems? Well, there’s all sorts of methods, but in this article I am going to discuss one technique I have found is simple, yet effective at mitigating curve fitting. Let’s get to it.
Note: All code will be provided at the bottom of the article
Common Methods Of Mitigating Curve Fitting:
Before I dig into the unique technique I have not seen discussed anywhere else, let’s do a quick review of common methods used to avoid curve fitting. In my opinion, there’s three methodologies of looking at this concept:
Method #1:
Avoiding curve fitting by doing some sort of validation/robustness/stress testing after the system is developed. This comes as one of the last steps in the process of system development and is essentially the last set of criteria the system needs to pass in order to be allocated real capital.
This is by far the most common way traders try to avoid curve fitting. A lot of times, traders will just try random combinations of rules and iterate over and over until they have a backtest equity curve they like. Then, they will run some robustness tests on that system to see if it passes.
While this may work, throwing random trading indicators at the wall and seeing what sticks is not ideal and definitely more prone to failing during robustness tests or during live trading. It’s highly likely these random combinations of trading rules are just fitting to noise, hence why the system fails during robustness testing.
An example of what these robustness tests might look like are a ±20% parameter variation sensitivity test, Monte Carlo tests, multi-market validations, etc.
Many systematic traders think that this is what robustness tests are for; to validate their soup of random indicators in a trading system. While robustness testing will help in determining if the system is real or not, it’s not guaranteed. This is why we also need sound system development practices, like discussed below in method #2.