The Overfitting Problem: Why Most Backtests Lie
An illustrative look at how parameter searches inflate apparent performance — and the validation habits that help.
This is a sample article demonstrating the intended layout of Quantopus research. It is not verified research and contains no real results.
Every researcher who tests enough ideas will eventually find one that looks exceptional by chance. The question is not whether this happens, but how to recognise it.
Multiple testing
When many variants of a strategy are evaluated on the same data, the best-performing variant is biased upward. Adjusting for the number of trials is a basic but frequently skipped step.
Out-of-sample discipline
Holding back data that is genuinely untouched, and only examining it once, is one of the simplest protections available. Walk-forward analysis extends the same idea through time.
Plausibility
A result should have an economic or behavioural explanation. Statistical significance without a mechanism deserves scepticism.