Your backtest shows a 20% maximum drawdown. You know that number is one path out of many, so you do the responsible thing: you run a Monte Carlo, reshuffle the trades a few thousand times, and read off the worst case. Say it comes back at 30%. You size the position so you can survive 30%, and you feel like you did the work.
Here is the question nobody asks. Was 30% right?
Not "is Monte Carlo a good idea". It is. Whether the number it hands you is the number reality respects. That is testable with nothing more than the strategy's own trades. I ran it on 3,418 strategies.
The simulated 1-in-20 worst case was breached 23.3% of the time.
TL;DR — the answer box
- The simulation's tail is too thin. The 1-in-20 worst-case drawdown was breached 23.3% of the time instead of 5%, 4.7x too often. Real drawdowns landed at the 28th percentile of the simulated distribution at the median.
- Returns are better calibrated than drawdowns, though neither is right. The simulated 1-in-20 return was breached 9.8% of the time against the 5% expected, which is far better calibrated than the drawdown at 23.3%.
- The method changes the answer by 27%. Reshuffling the trade order, which is what most retail platforms ship, puts the 1-in-20 drawdown at 1.32x the backtest's. Bootstrapping puts it at 1.67x.
- In this bootstrap sample, the median case ran roughly two-thirds worse than the backtest. Median 1-in-20 drawdown: 1.66x the backtested figure, 2.30x at the 90th percentile of strategies.
- You can carry far less size than you think. If your tolerance is a 20% drawdown, the median strategy could safely carry 0.25x of a full unit, and only 12.5% could carry a full one.
How I tested
Eight markets on daily bars (SPY, QQQ, IWM, DIA and ES, NQ, NG, JY futures) and the same three strategy families used across this pillar: RSI mean reversion, moving-average trend, Donchian breakout. Of the 5,472 parameter variants, 3,418 met the study's admission floor of at least 30 trades in the development period, the first 75% of each market's history. Admission reads the development period only, so how many trades a variant went on to take in the untouched final 25% plays no part. Median development trade count: 91.
The resampling itself is the ordinary Monte Carlo method: draw from a distribution enough times and the spread of the draws stands in for the spread of outcomes you might have had. Every simulation is seeded and reproducible. Trades come from the engine's vectorised backtester — signal at the close, entry at the next open, long only, one unit, frictionless.
Throughout, "the 1-in-20 case" means the 5th percentile of the simulated distribution: the outcome you should expect to see roughly one run in twenty. That is the number a trader actually uses for sizing, so that is the number worth checking.
The test nobody runs
Here is the design, because the design is the whole contribution.
For each strategy, take only its development trades, the ones from the first 75% of history. Resample them into 1,000 alternative paths, each as long as the development trade rate predicts for the final 25%: development trades per bar, times the bars in that slice. That length is known before the final 25% begins, and the median path is 30 trades. Now you have a forecast: a full distribution of drawdowns the strategy "should" produce over that many trades.
Then look at what it actually did in that final 25%, and find where the real drawdown falls inside the forecast. Grading a procedure on a slice of data it never saw is the standard way to test one; the unusual part here is doing it to a simulation's own forecast rather than to a strategy's returns.
If Monte Carlo is calibrated, those percentiles should be uniform. Reality should land below the 5th percentile exactly 5% of the time.
| Expected if calibrated | Actual | |
|---|---|---|
| Real drawdown worse than the simulated 1-in-20 | 5% | 23.3% |
| Real drawdown worse than the simulated median | 50% | 66.5% |
| Median percentile where reality landed | 50th | 28th |
Nearly five times too often. The thing you built specifically to stop being surprised is surprising you once every four or five strategies.
A calibrated simulation makes that histogram flat. This one has a wall at the bad end.
It holds across every family: 26.4% for mean reversion, 25.1% for trend, 17.0% for breakout. Breakout is the least bad, and it is still more than three times the rate it should be.
Why. Not a bug — an assumption. Resampling treats each trade as independent, as though the outcome of one tells you nothing about the next. Real losing trades do not arrive that way; they arrive in runs. Shuffling the order destroys whatever clustering is there, which is one candidate explanation for why the simulated drawdown ran shallow. The limit to state: this test cannot separate how much of the miss is trade clustering and how much is the market regime changing between the two periods.
And note the asymmetry, because it tells you where the assumption bites: the simulated 1-in-20 return was breached only 9.8% of the time, and reality's median return percentile was 58th, slightly better than the simulation expected. Monte Carlo gets the destination roughly right and the road badly wrong. Drawdown is a property of ordering. Ordering is the one thing the method throws away.
Which button did you press?
Before the fix, one more thing worth knowing: "run a Monte Carlo" is not one thing.
| Method | 1-in-20 drawdown as a multiple of the backtest |
|---|---|
| Reshuffle trade order (permutation) | 1.32x |
| Block bootstrap (keeps runs of 5) | 1.59x |
| Bootstrap with replacement | 1.67x |
Same strategies, same 2,000 simulations, three standard methods, and a 27% spread in the answer.
Reshuffling is the most optimistic, and it is the one most retail platforms ship. The reason is structural: a permutation reorders the same trades, so every simulated run ends at exactly the same final equity as the backtest. You are only sampling paths, never sampling outcomes. Bootstrapping with replacement, the standard statistical way to resample a sample, lets the sample itself vary, which is the honest version if you believe your trades are a sample of something rather than the population.
Given the calibration result, the ranking matters: the most common method is also the one that produces the shallowest 1-in-20 estimate of the three tested.
How much worse than the backtest?
Using the bootstrap, across 1,000 strategies:
| Value | |
|---|---|
| Median 1-in-20 drawdown vs the backtest | 1.66x |
| 90th percentile of strategies | 2.30x |
| Backtest's own drawdown, as a percentile of simulated outcomes | 43rd |
| Strategies with at least one losing simulated run | 95.8% |
| Strategies losing money in more than 1 run in 20 | 64.0% |
Two useful things here.
The backtest's own drawdown sits at the 43rd percentile of the simulated distribution. It is not a worst case. It is a near-typical case, with 43% of alternative histories producing a deeper drawdown, which means the drawdown you have been staring at and steeling yourself for is roughly the middle of the range.
And 64.0% of these strategies lost money in more than one simulated run in twenty. That is the real content of a Monte Carlo: not a scarier drawdown number, but the share of alternative histories in which the same strategy ends up losing money.
The question you were actually asking
Nobody runs a Monte Carlo out of curiosity. They run it to decide how much money to put behind the thing. So here is that answer directly.
For each allocation multiple, scale every trade return by it and recompute the simulated drawdown. The safe allocation is the largest multiple that keeps the chance of breaching your tolerance at 1 in 20 or better.
| If you can tolerate | Median safe allocation | Safe at full size | No safe size at all |
|---|---|---|---|
| A 10% drawdown | 0.00x | 1.0% | 59.0% |
| A 20% drawdown | 0.25x | 12.5% | 10.5% |
| A 30% drawdown | 0.50x | 27.8% | 0.2% |
At a 20% tolerance, the number traders say out loud most often, the median strategy carries a quarter of a full unit. Only one strategy in eight can carry a full one. Across the 400 variants, the median probability of breaching 20% at full allocation was 60.9%.
At a 10% tolerance, 59.0% of these strategies have no safe size at or above 0.25x a full unit, which is the smallest allocation the study put on the grid. Below that the study did not test, so the finding is that nothing on the tested grid was safe, not that no size exists.
This is where Monte Carlo earns its place. Not as a scary number generator, but as the step that turns "this strategy looks good" into "this strategy gets 25% of a unit." And now apply the calibration finding on top: the simulated 1-in-20 worst case was breached 23.3% of the time, 4.7x the rate it advertises. These allocations come from the same simulation, and the study did not rerun the sizing on a tail widened to match that miss, so treat them as what the tested model says is safe, not as a calibrated safe size adjusted for the miss.
Two housekeeping answers
How many simulations? About a thousand.
| Draws | Median error vs a 10,000-draw reference |
|---|---|
| 100 | 4.0% |
| 1,000 | 1.5% |
| 5,000 | 0.8% |
Above 1,000 you are buying decimal places. If your platform takes ten minutes to run however many draws it defaults to, that time is buying decimal places.
Does a longer track record help? For the estimate, yes. For the outcome, no.
| Development trades | 1-in-20 drawdown vs the backtest |
|---|---|
| 30-49 | 1.76x |
| 50-99 | 1.67x |
| 100-199 | 1.58x |
| 200-399 | 1.67x |
| 400+ | 1.75x |
No trend, anywhere. This one surprised me — I expected the ratio to tighten with sample size. It does not. More trades make your estimate of the distribution more reliable; they do not make the distribution narrower. Across the study's trade-count buckets the median 1-in-20-to-backtest ratio showed no trend.
The verdict — and the honest limits
Monte Carlo is worth running. Just do not read its 1-in-20 drawdown as a 1-in-20.
What it does well. It tells you how often a strategy loses across alternative histories, with 64.0% of these strategies losing in more than 1 run in 20, and it converts a backtest into a position size, which nothing else on the report does. Both of those are load-bearing.
What it does badly. Drawdowns. The 1-in-20 case was breached 23.3% of the time against the 5% advertised. It underestimates the depth of trouble because it assumes trouble does not cluster, and trouble is the one thing that always clusters.
How to use it anyway. This study never tested which of the three resampling methods is closest to the truth. It only shows that bootstrap gives the least optimistic (most conservative) of the three 1-in-20 estimates. Absent that accuracy test, size against the conservative one: use bootstrap. Run a thousand draws. Take the 1-in-20 drawdown and relabel the threshold: it was breached 23.3% of the time in this calibration sample, so what the software calls a 1-in-20 outcome behaved closer to a 1-in-4. The study did not rerun its sizing on that relabelled threshold, so read it as a warning about the allocation numbers rather than a replacement for them.
Limits. Long only, one unit, no leverage, no shorting, no stops, no position sizing beyond the allocation multiplier. Frictionless — costs deepen every drawdown, simulated and real alike. The allocation analysis scales trade returns linearly, which is the standard approximation and ignores margin rules, borrowing cost and gap risk. And the calibration test compares a development-period trade distribution against a later period, so some of the miss is regime change rather than trade clustering; this study does not separate the two, and an honest reading is that both contribute. Every result is a property of these eight markets over these date ranges.
What this means for you
- Bootstrap, don't reshuffle. 1.67x versus 1.32x. The default button is the optimistic one.
- Run 1,000 draws and stop. 1.5% median error against a 10,000-draw reference, and 3.7% at the 90th percentile of the 200 variants tested.
- Read the backtest drawdown as a near-typical outcome, not a worst case. It sits at the 43rd percentile of simulated outcomes.
- Treat the simulated 1-in-20 bound as weaker than it advertises. Across 3,418 variants it was breached 23.3% of the time, not 5%.
- Check the share of losing runs, not just the drawdown. 64.0% of these strategies lost money in more than one run in twenty.
- Convert the output into an allocation. At a 20% tolerance the median strategy carries 0.25x — and that is before adjusting for the calibration miss.
- Stop expecting more trades to tighten the tail. The ratio was flat from 30 trades to 400-plus.
This is one spoke of the robustness testing guide. The others cover how to configure a walk-forward and which backtest metric actually predicts anything.
Running this on your own equity curve is one of the things AlgoChef does with an imported backtest — five resampling methods, including the bootstrap this study ended up recommending.
