Your backtest shows a 20% maximum drawdown. You know that number is one path out of many, so you do the responsible thing: you run a Monte Carlo, reshuffle the trades a few thousand times, and read off the worst case. Say it comes back at 30%. You size the position so you can survive 30%, and you feel like you did the work.
Here is the question nobody asks. Was 30% right?
Not "is Monte Carlo a good idea" — it is. Whether the number it hands you is the number reality respects. That is testable, and it takes about ninety seconds of compute once you have the trades. I ran it on 3,232 strategies.
The simulated 1-in-20 worst case was breached 22.5% of the time.
TL;DR — the answer box
- The simulation's tail is too thin. The 1-in-20 worst-case drawdown was breached 22.5% of the time instead of 5% — 4.5x too often. Real drawdowns landed at the 27th percentile of the simulated distribution at the median.
- Returns are better calibrated than drawdowns, though neither is right. The simulated 1-in-20 return was breached 9.5% of the time against the 5% expected, which is far better calibrated than the drawdown at 22.5%.
- The method changes the answer by 24%. Reshuffling the trade order — what most retail platforms ship — puts the 1-in-20 drawdown at 1.33x the backtest's. Bootstrapping puts it at 1.65x.
- Plan for roughly two-thirds worse than the backtest. Median 1-in-20 drawdown: 1.66x the backtested figure, 2.35x at the 90th percentile of strategies.
- You can carry far less size than you think. If your tolerance is a 20% drawdown, the median strategy could safely carry 0.25x of a full unit, and only 9.8% could carry a full one.
How I tested
Eight markets on daily bars — SPY, QQQ, IWM, DIA and ES, NQ, NG, JY futures — and the same three strategy families used across this pillar: RSI mean reversion, moving-average trend, Donchian breakout. Of the 5,472 parameter variants, 3,232 had enough trades to simulate honestly: at least 30 in the development period (the first 75% of each market's history) and at least 10 in the untouched final 25%. Median development trade count: 96.
Every simulation is seeded and reproducible. Trades come from the engine's vectorised backtester — signal at the close, entry at the next open, long only, one unit, frictionless.
Throughout, "the 1-in-20 case" means the 5th percentile of the simulated distribution: the outcome you should expect to see roughly one run in twenty. That is the number a trader actually uses for sizing, so that is the number worth checking.
The test nobody runs
Here is the design, because the design is the whole contribution.
For each strategy, take only its development trades — the ones from the first 75% of history. Resample them into 1,000 alternative paths, each exactly as long as the number of trades the strategy went on to take in the final 25%. Now you have a forecast: a full distribution of drawdowns the strategy "should" produce over that many trades.
Then look at what it actually did in that final 25%, and find where the real drawdown falls inside the forecast.
If Monte Carlo is calibrated, those percentiles should be uniform. Reality should land below the 5th percentile exactly 5% of the time.
| Expected if calibrated | Actual | |
|---|---|---|
| Real drawdown worse than the simulated 1-in-20 | 5% | 22.5% |
| Real drawdown worse than the simulated median | 50% | 68.2% |
| Median percentile where reality landed | 50th | 27th |
Four and a half times too often. The thing you built specifically to stop being surprised is surprising you once every four or five strategies.
A calibrated simulation makes that histogram flat. This one has a wall at the bad end.
It holds across every family: 24.6% for mean reversion, 26.2% for trend, 17.3% for breakout. Breakout is the least bad, and it is still three and a half times the rate it should be.
Why. Not a bug — an assumption. Resampling treats each trade as independent, as though the outcome of one tells you nothing about the next. Real losing trades do not arrive that way. They arrive in runs, because the conditions that produce them persist. Shuffling the order breaks exactly the clustering that makes real drawdowns deep. The limit to state: this test cannot separate how much of the miss is trade clustering and how much is the market regime changing between the two periods.
And note the asymmetry, because it tells you where the assumption bites: the simulated 1-in-20 return was breached only 9.5% of the time, and reality's median return percentile was 57th — slightly better than the simulation expected. Monte Carlo gets the destination roughly right and the road badly wrong. Drawdown is a property of ordering. Ordering is the one thing the method throws away.
Which button did you press?
Before the fix, one more thing worth knowing: "run a Monte Carlo" is not one thing.
| Method | 1-in-20 drawdown as a multiple of the backtest |
|---|---|
| Reshuffle trade order (permutation) | 1.33x |
| Block bootstrap (keeps runs of 5) | 1.60x |
| Bootstrap with replacement | 1.65x |
Same strategies, same 2,000 simulations, three standard methods — and a 24% spread in the answer.
Reshuffling is the most optimistic, and it is the one most retail platforms ship. The reason is structural: a permutation reorders the same trades, so every simulated run ends at exactly the same final equity as the backtest. You are only sampling paths, never sampling outcomes. Bootstrapping with replacement lets the sample itself vary, which is the honest version if you believe your trades are a sample of something rather than the population.
Given the calibration result, the ranking matters: the most common method is also the one that produces the shallowest 1-in-20 estimate of the three tested.
How much worse than the backtest?
Using the bootstrap, across 1,000 strategies:
| Value | |
|---|---|
| Median 1-in-20 drawdown vs the backtest | 1.66x |
| 90th percentile of strategies | 2.35x |
| Backtest's own drawdown, as a percentile of simulated outcomes | 42nd |
| Strategies with at least one losing simulated run | 94.7% |
| Strategies losing money in more than 1 run in 20 | 61.6% |
Two useful things here.
The backtest's own drawdown sits at the 42nd percentile of the simulated distribution. It is not a worst case. It is a near-typical case, with 42% of alternative histories producing a deeper drawdown, which means the drawdown you have been staring at and steeling yourself for is roughly the middle of the range.
And 61.6% of these strategies lost money in more than one simulated run in twenty. That is the real content of a Monte Carlo: not a scarier drawdown number, but the share of alternative histories in which the same strategy ends up losing money.
The question you were actually asking
Nobody runs a Monte Carlo out of curiosity. They run it to decide how much money to put behind the thing. So here is that answer directly.
For each allocation multiple, scale every trade return by it and recompute the simulated drawdown. The safe allocation is the largest multiple that keeps the chance of breaching your tolerance at 1 in 20 or better.
| If you can tolerate | Median safe allocation | Safe at full size | No safe size at all |
|---|---|---|---|
| A 10% drawdown | 0.00x | 0.2% | 62.5% |
| A 20% drawdown | 0.25x | 9.8% | 12.0% |
| A 30% drawdown | 0.50x | 25.8% | 0.8% |
At a 20% tolerance — the number traders say out loud most often — the median strategy carries a quarter of a full unit. Not one strategy in ten can carry a full one. Across the 400 variants, the median probability of breaching 20% at full allocation was 61.9%.
At a 10% tolerance, 62.5% of these strategies have no safe size at or above 0.25x a full unit, which is the smallest allocation the study put on the grid. Below that the study did not test, so the finding is that nothing on the tested grid was safe, not that no size exists.
This is where Monte Carlo earns its place. Not as a scary number generator, but as the step that turns "this strategy looks good" into "this strategy gets 25% of a unit." And now apply the calibration finding on top: the simulated 1-in-20 worst case was breached 22.5% of the time, roughly 4.5x too optimistic in frequency, so treat these allocations as ceilings, not targets.
Two housekeeping answers
How many simulations? About a thousand.
| Draws | Median error vs a 10,000-draw reference |
|---|---|
| 100 | 4.3% |
| 1,000 | 1.3% |
| 5,000 | 0.7% |
Above 1,000 you are buying decimal places. If your platform takes ten minutes to run however many draws it defaults to, that time is buying decimal places.
Does a longer track record help? For the estimate, yes. For the outcome, no.
| Development trades | 1-in-20 drawdown vs the backtest |
|---|---|
| 30–49 | 1.80x |
| 50–99 | 1.67x |
| 100–199 | 1.59x |
| 200–399 | 1.71x |
| 400+ | 1.77x |
No trend, anywhere. This one surprised me — I expected the ratio to tighten with sample size. It does not. More trades make your estimate of the distribution more reliable; they do not make the distribution narrower. Across the study's trade-count buckets the median 1-in-20-to-backtest ratio showed no trend.
The verdict — and the honest limits
Monte Carlo is worth running. Its output is worth about half of what you think.
What it does well. It tells you how often a strategy loses across alternative histories, with 61.6% of these strategies losing in more than 1 run in 20, and it converts a backtest into a position size, which nothing else on the report does. Both of those are load-bearing.
What it does badly. Drawdowns. The 1-in-20 case was breached 22.5% of the time against the 5% advertised. It underestimates the depth of trouble because it assumes trouble does not cluster, and trouble is the one thing that always clusters.
How to use it anyway. Bootstrap rather than reshuffle. Run a thousand draws. Take the 1-in-20 drawdown and relabel the threshold: it was breached 22.5% of the time in this calibration sample, so what the software calls a 1-in-20 outcome behaved closer to a 1-in-4. The study did not rerun its sizing on that relabelled threshold, so read it as a warning about the allocation numbers rather than a replacement for them.
Limits. Long only, one unit, no leverage, no shorting, no stops, no position sizing beyond the allocation multiplier. Frictionless — costs deepen every drawdown, simulated and real alike. The allocation analysis scales trade returns linearly, which is the standard approximation and ignores margin rules, borrowing cost and gap risk. And the calibration test compares a development-period trade distribution against a later period, so some of the miss is regime change rather than trade clustering; this study does not separate the two, and an honest reading is that both contribute. Every result is a property of these eight markets over these date ranges.
What this means for you
- Bootstrap, don't reshuffle. 1.65x versus 1.33x — the default button is the optimistic one.
- Run 1,000 draws and stop. 1.3% median error against a 10,000-draw reference.
- Read the backtest drawdown as a near-typical outcome, not a worst case. It sits at the 42nd percentile of simulated outcomes.
- Treat the simulated 1-in-20 bound as weaker than it advertises. Across 3,232 variants it was breached 22.5% of the time, not 5%.
- Check the share of losing runs, not just the drawdown. 61.6% of these strategies lost money in more than one run in twenty.
- Convert the output into an allocation. At a 20% tolerance the median strategy carries 0.25x — and that is before adjusting for the calibration miss.
- Stop expecting more trades to tighten the tail. The ratio was flat from 30 trades to 400-plus.
This is one spoke of the robustness testing guide. The others cover how to configure a walk-forward and which backtest metric actually predicts anything.
Running this on your own equity curve is one of the things AlgoChef does with an imported backtest — five resampling methods, including the bootstrap this study ended up recommending.
