Overfitting in Trading: How to Tell If Your Backtest Lies

If you tested 200 settings and kept the best one, treat that best one as mostly luck until it proves otherwise on data it has never seen. That’s the practical core of overfitting in trading: the more variations you try, the better your best backtest looks, whether or not there’s any edge underneath. The glossary entry on overfitting has the definition; this post is about catching it in your own results before the market does.

Last updated: November 2026.

A hypothetical: 200 EMA settings on Nifty futures

Say you test an EMA crossover on Nifty futures. The fast EMA runs from 5 to 24 (20 values) and the slow EMA from 30 to 120 in steps of 10 (10 values). That’s 200 combinations. You run all of them over five years of daily data, sort by Sharpe ratio, and the winner, fast 13 and slow 70, shows an annualised Sharpe of 1.3. These numbers are invented to illustrate the statistics; they aren’t results from a real test.

A Sharpe of 1.3 feels like a discovery. The question that matters is how good the best of 200 would look if none of them had an edge at all.

Why the best of 200 looks good by luck

Every backtest Sharpe is an estimate, and estimates have noise. For a strategy with no real edge, the standard error of an annualised Sharpe over T years is roughly 1/√T. Over five years that’s about 0.45, so a strategy with zero true edge will show a five-year Sharpe anywhere from about −0.9 to +0.9 most of the time, purely by chance.

Now take the best of many. If the 200 trials were genuinely independent ideas, the best one would show a Sharpe of about 1.2 on average with no edge anywhere in the set. You can check this in a few lines:

import numpy as np

rng = np.random.default_rng(7)
days, trials, runs = 5 * 250, 200, 500
best = []
for _ in range(runs):
    r = rng.normal(0, 0.01, size=(trials, days))       # zero-edge daily returns
    sharpe = r.mean(axis=1) / r.std(axis=1) * np.sqrt(250)
    best.append(sharpe.max())
print(round(np.mean(best), 2))                         # prints about 1.2

So your 1.3 barely clears what pure noise produces. Bailey, Borwein, López de Prado and Zhu made the same point more formally in Pseudo-Mathematics and Financial Charlatanism: with only five years of data, trying more than about 45 independent configurations almost guarantees you’ll find one with an in-sample Sharpe of 1 and an expected out-of-sample Sharpe of zero.

One honest wrinkle: an EMA grid is not 200 independent ideas. Fast 13 and fast 14 with the same slow EMA trade almost identically, so the grid behaves like far fewer separate trials. In a quick simulation of this exact 20 × 10 grid on random-walk prices (no edge by construction, no costs), the best Sharpe averaged about 0.4. That’s less flattering than 1.2, but about one run in twenty still produced a best above 1.0 from pure noise.

The bigger problem is everything outside the grid. Before this test you probably tried RSI, then Supertrend, then 15-minute bars, then Bank Nifty, then a version that skips expiry days. Those are much closer to independent trials, and they add up fast. The grid is only the part of your research you remember.

Plateau or spike? Read the parameter heatmap

Put fast EMA on one axis and slow EMA on the other, and colour each cell by Sharpe or net profit after costs. Whatever tool you use, look at the grid this way; a sorted list hides the one thing you need to see.

A plateau is a broad warm patch. The best cell sits among neighbours that are nearly as good, so moving the fast EMA from 13 to 11 or 16 changes the result a little, not a lot. That suggests the strategy is responding to something real about how the market trends.

A spike is one bright cell surrounded by mediocre or losing ones, or a checkerboard of good and bad cells with no pattern. That’s the signature of fitting noise.

My working check: compare the best cell with the average of its eight neighbours. If the neighbours average less than about half of the best, I treat the peak as noise and either pick the centre of the plateau or drop the idea. If there’s no plateau anywhere on the map, there’s no strategy, just a lucky cell.

One caution. A plateau can still be built on a handful of trades. If a wide band of slow EMAs all made their money from the same two or three big trend moves, the plateau is real and still fragile. Check what share of total profit comes from the top five trades.

Keep a holdout you only look at once

Before you start, set aside the most recent stretch of data and don’t touch it. If you have history from 2019 to October 2026, you might build on 2019 to 2024 and lock 2025 and 2026 away.

The rules are strict because they have to be. Decide the split before you see any results and run the frozen strategy on the holdout once. If you look, dislike it, tweak something and look again, the holdout has become in-sample data. Expect it to be worse than the in-sample period; some decay is normal. A collapse, or a flip from profit to loss, is the warning you were looking for.

Walk-forward: the holdout, repeated

A single holdout is one roll of the dice. Walk-forward analysis repeats it: optimise on, say, 24 months, trade the next 6 months with the chosen parameters, roll everything forward 6 months, and repeat. Stitch the out-of-sample pieces together and that stitched curve is a far more honest estimate than any single backtest. The walk-forward optimization glossary entry has the mechanics.

Two things to look at. First, the stitched out-of-sample curve itself. Second, how much the chosen parameters jump between windows. If the optimiser picks 8/40 in one window, 21/110 in the next and 6/90 after that, there’s no stable optimum, which is another way of saying there’s no edge the parameters are tracking.

Some practitioners compare out-of-sample to in-sample annualised returns and use about 50% as a rough bar. I treat that as a smoke alarm rather than a pass mark. And note that you can overfit walk-forward too, by trying ten window lengths and reporting the one that worked.

Monte Carlo: useful, but not for this question

Monte Carlo analysis that reshuffles the order of your trades is very good at one thing: showing the range of drawdowns and losing streaks your trade list could have produced. That’s essential for sizing.

What it can’t do is tell you whether the trades themselves came from a real edge. Reshuffle an overfit trade list a thousand times and you get a tidy, confident distribution of a fake edge. Run Monte Carlo after you believe the edge is real. Variants that perturb the inputs, such as nudging every parameter by 10% or randomly dropping a fifth of the trades, are more useful as an overfitting check.

Count your trials honestly

Every decision you made after seeing a result is a trial. Each parameter value, each filter you added and later removed, each timeframe, each instrument, each “what if I exclude expiry days”, each date range you started from. If a result changed what you tried next, it counts.

The cheapest habit in this whole post is a research log. A spreadsheet with the date, what you changed and what happened is enough. Without one, you can’t estimate how many trials you ran, and every statistical defence below depends on that number.

The Deflated Sharpe Ratio, in plain terms

Bailey and López de Prado’s Deflated Sharpe Ratio (Journal of Portfolio Management, 2014) formalises everything above. Instead of asking “is my Sharpe above zero?”, it asks “is my Sharpe above what the best of my N trials would show by luck?”

It adjusts for how many trials you ran, how much their Sharpe ratios varied, how long your track record is, and whether returns are skewed or fat-tailed, and gives you a probability that the true Sharpe is above zero once selection is accounted for.

Two consequences matter for retail traders. Because it uses the spread of Sharpe ratios across trials, a grid of near-identical settings is penalised less than 200 genuinely different ideas, which is fair, but only if you recorded every trial. And because it penalises negative skew, option-selling strategies, with their small steady gains and occasional big losses, take a further haircut. Their smooth equity curves overstate the edge, and the maths says so.

Rules of thumb for trades per parameter

These are my working thresholds, not laws:

• At least 100 trades in the in-sample period before I read anything into a Sharpe ratio.
• Roughly 30 or more trades per optimised parameter, so a three-parameter strategy needs 90+ before I’m comfortable, and the 100 floor still applies.
• No more than two or three optimised parameters for a retail strategy. Things like session times or cost assumptions should be fixed by logic, not optimised.
• At least 30 out-of-sample trades, whether in one holdout or across all walk-forward test windows combined. Fewer than that and the out-of-sample test can’t tell you much.

Costs: where overfit strategies hide

Always optimise after costs. Faster settings trade more, and on NSE every round trip has a real price. One lot of Nifty futures at 25,000 is ₹16.25 lakh of notional (25,000 × 65, lot size as of 2026). At a ₹20-per-order broker, the round trip costs about ₹966: roughly ₹812 of STT on the sell side, ₹60 of exchange charges, ₹40 of brokerage, ₹32 of stamp duty and about ₹22 of GST and SEBI fees. That’s close to 15 Nifty points before slippage. Check your own broker on the brokerage calculator.

An optimiser run before costs can’t see that fast, busy settings pay that ₹966 every time they trade, so its favourites often include settings whose gross edge is smaller than their costs. A simple stress test: double the cost and slippage assumptions and rerun. A real edge thins out. An overfit one usually turns negative.

We build INDfolio, so weigh this accordingly. Its parameter optimisation, walk-forward and Monte Carlo tools exist for exactly these checks, and backtests include the Indian cost stack. None of it will stop you running 2,000 combinations and believing the winner. That discipline is yours. For what a backtester itself needs to get right before any of this matters, see our piece on institutional-grade backtesting in India.

A short smell test

Before you trust a backtest, yours or anyone’s, check that:

• You can say roughly how many variations you tried.
• The chosen parameters sit on a plateau, not a spike.
• There are at least 100 trades, and the top five aren’t most of the profit.
• It’s profitable after full costs and still positive with costs doubled.
• Out-of-sample or walk-forward results are worse but still positive.
• No single year or single event carries the whole result.
• Every rule has a reason you could have written down before seeing the data.
• You’ve looked at the holdout once.

If an equity curve is so smooth that it looks drawn with a ruler, assume a bug or a fit before you assume genius.

Frequently asked questions

What is overfitting in trading? It’s tuning a strategy so closely to past data that it captures that period’s noise rather than a repeatable edge. The backtest looks excellent and live results disappoint.

How do I know if my trading strategy is overfitted? Look for a spike rather than a plateau in the parameter heatmap, a big drop on out-of-sample data, profits concentrated in a few trades, and results that vanish when costs are doubled. Any one of these is a warning; two or more usually settle it.

Is curve fitting the same as overfitting? In trading, yes. Both describe shaping rules and parameters around historical data until the backtest fits the past better than the future will.

How many trades do I need for a reliable backtest? I wouldn’t read much into fewer than 100 in-sample trades, and I’d want roughly 30 per optimised parameter. Fewer trades means a wider range of Sharpe ratios that luck alone can produce.

Does walk-forward analysis prevent overfitting? It reduces it a lot, because every result it reports is out-of-sample. But you can still overfit the walk-forward itself by trying many window lengths and keeping the one that looks best.

What is a good Sharpe ratio for a backtest? It depends on how many variations you tried and how long the test is. A Sharpe of 1.2 over five years is about what the best of 200 independent no-edge strategies shows, so judge it against your trial count, not against zero. The Sharpe ratio calculator shows the basic arithmetic.