Curve fitting in trading: why your backtest lies to you
Curve fitting versus discovery: how iterative tweaking creates false confidence, why degrees of freedom explode silently, and how walk-forward splits plus preregistered metrics push back.
Curve fitting trading strategy and trading system curve fitting test searches point to the same failure mode: you adjust a model until it hugs historical prices. The backtest looks brilliant because you optimized on the path you judge.
This is not "being smart about data." It is training a camera to photograph yesterday.
How curve fitting sneaks in
- Manual tweaks after peeking at equity curves
- Parameter searches without a held-out protocol (Parameter optimization)
- Many strategy variants on one dataset (Data snooping)
- "Just one more filter" after you saw the drawdown in window 3
- reusing the same narrative across different symbols without independent evidence
Degrees of freedom are not only parameters
Anything you change after observing results consumes degrees of freedom:
- indicator windows
- universe selection rules
- start dates and warm-up policies
- fee assumptions after you did not like the first answer
If you do not budget these choices explicitly, you will always feel honest while overfitting.
What to do instead
- Fix a research protocol before touching parameters (write the success metrics first)
- Use walk-forward discipline (What is WFA?)
- Treat hyperopt winners as suspects (Hyperopt)
- Run cost stress tiers and record whether rankings are stable
The emotional trap
Curve fitting feels like progress because the chart improves immediately. Real discovery is slower: it shows up as robustness under stress, not as a prettier backtest line.
A simple test
Ask: "If I freeze parameters and roll time forward, does the edge still exist?" If the answer requires more tuning, you are still fitting (What is WFA?).
A stronger test (preregistered outcomes)
Before you optimize, write down:
- which metrics decide pass/fail
- what threshold counts as pass
- how many variants you are allowed to try
If you change the metric after you see results, reset the protocol.