ForecastingStocks

Overfitting is the central villain of quantitative finance. It is the reason the gap between backtest and live results is so wide, so reliably, that experienced quants assume a strategy will perform worse live before they assume anything else.

What overfitting is

A model overfits when it learns the noise in the training data instead of the underlying signal. It describes the past in exquisite detail and predicts the future poorly, because next year's noise is different. The more knobs a model has — parameters, features, rules — the more easily it memorises noise.

Data-snooping: the subtle epidemic

Even with clean train/test splits, you can overfit through the research process itself. This is data-snooping (or backtest overfitting):

You test 1,000 strategy variations and pick the best one. By pure chance, some will look brilliant on any dataset — and you have selected the luckiest, not the best.
You tweak a strategy repeatedly against the same test set until it passes. The test set is now contaminated; it has effectively become training data.
The whole field tries thousands of ideas on the same historical data; the survivors that get published are disproportionately the lucky ones.

The more hypotheses you try, the more a backtest's apparent significance is meaningless. A Sharpe ratio that would be impressive from one test is unremarkable as the best of a thousand.

Defending yourself

Prefer simple models with few parameters. Simplicity is robustness.
Demand an economic rationale. A pattern with a credible reason to exist survives; a pure data artefact does not.
Hold out a final test set you touch exactly once, after all decisions are made.
Adjust for the number of trials — be far more sceptical of the best of many than of a single pre-registered hypothesis.
Assume decay. Even a real edge weakens as it is discovered and crowded.

The professional mindset is not 'how good is this backtest?' but 'how am I fooling myself?' That suspicion, applied relentlessly, is the difference between a quant who survives and one who blows up.