TL;DR: the answer box
- Trading is a skill, and the part you can measure is telling an edge from luck. Spotting patterns is the easy part. Luck finds patterns too.
- Luck backtests well. 5,000 random rules, mined on 10 datasets with no edge: the median best made 6.4% a year at a Sharpe ratio of 0.65. In the next period, those winners made a median -1.2%.
- Real data found more than luck did. On real SPY, the best rule showed 11.2% a year. The real search beat the luck search by 4.8% a year. That gap is the real effect.
- Win rate did not separate the rule from a coin flip. Random entries won 61.25% of trades. A public RSI(2) rule won 60.34%. On return over worst drawdown, the rule led: 0.0767 against 0.0494.
- The test is a checklist, not a gift. Of 5,472 strategy variants, 63.7% made money in the final quarter of their history. Of the variants that passed at least 10 of 11 checks, 98.0% made money there. Beating buy and hold stayed rare: 20.7% of those, against 27.8% of all variants.
Is trading a skill or luck?
Most people asking this want to know if great traders are born. A feel for the market. A gift for reading charts.
Here is the problem with that picture. A gift for spotting patterns is exactly what luck rewards. Mine 5,000 rules on data with no edge, and the median best still shows 6.4% a year. So never stop at "did it work?" Ask instead: "would it have worked on data where nothing can work?"
I learned that the expensive way. I entered the markets in 1998, at the dot-com peak. Over the next years I lost roughly $270K to overfitting and leverage. An overfit system is luck dressed up as skill. I rebuilt on the 2014 Turtle Traders lesson: simple systems plus proper risk management.
So my answer is flat. Trading is a skill, and the part you can test is how much of a result is luck.
Think of a coin flip. If you cannot say how your trades differ from a coin deciding when to buy, you do not know yet if you have a skill. That coin runs through the rest of this page.
How I answered it
I ran no new backtest for this article, so no execution rule applies to it. Every measured number comes from two studies already on this site, and you can open both:
- Robustness testing. 5,472 strategy variants across 8 markets and 3 strategy families. Plus 5,000 random rules mined on SPY, then mined again on 10 datasets with no edge. It reads each signal at the close and buys the next open, long only, one unit.
- AI strategy validation. SPY daily data from 1993 to 2026, 8,398 bars, with a seeded coin flip on the last 10 years.
A few plain words first:
- Backtest: running a rule on past prices to see what it would have made.
- In sample: the history a rule was picked on. Out of sample: history held back to test it afterwards.
- Sharpe ratio: return divided by how bumpy the ride was. Higher looks smoother and better.
- Drawdown: the biggest drop from a high point.
- Buy and hold: buying SPY and doing nothing else. Every rule here has to beat it.
- Median: the middle value, or the typical result.
No-edge data is the first tool. It is how you measure luck.
The seeded coin flip is the second tool. It buys on random days, as often as the strategies did, and holds each trade for 5 bars. It is averaged over 10 seeds. Seeded means it gives the same answer every time you run it.
What does luck look like in a backtest?
Better than you think.
The robustness study took 221 ordinary trading conditions, such as oscillator levels, moving averages and breakouts. Seven of them were meaningless on purpose: calendar conditions with no reason to work. It mixed them into 5,000 random rules of one to three conditions each. Then it mined them twice. Once on real SPY. Once on each of the 10 no-edge datasets.
Look at the orange bars. 6.4% a year at a Sharpe ratio of 0.65, on data where the true edge is exactly zero. The best single no-edge result reached 10.4% a year. If a stranger sold you that backtest, you could not tell it from skill by looking at it.
Then those winners met the next period. Median: -1.2%.
More tries make luck look better too. On no-edge data, testing 1 RSI rule gave a median Sharpe of 0.00. Testing 384 gave 0.36. Nothing got smarter. There were just more lottery tickets. Harvey and Liu make the same point in Backtesting, in the Journal of Portfolio Management: the more strategies you test, the bigger the discount a backtest's Sharpe ratio needs.
Why does a short track record fool you?
Because luck has an easier job over a short stretch.
On 3 years of no-edge data, testing 384 RSI rules found a median best of 9.2% a year, at a Sharpe of 1.17. At the luckiest 1 in 20, ranked on each measure on its own, the best rule reached 20.8% a year, and the best Sharpe reached 1.78. On 20 years, the same search found only 3.1% at 0.46.
A trader with a great few years looks gifted. So does a 3-year backtest. Ask both the same question: what would luck have made over the same length, with the same number of tries?
Is there real skill at all?
Yes. On real SPY, the search found more than luck did, and less than its backtest showed.
The best rule on real SPY was simple: buy when RSI(2) drops below 15, sell 5 days later. RSI(2) is the 2-day Relative Strength Index, a short-term gauge of how oversold a market is. In sample it made 11.2% a year at a Sharpe of 0.79, over 339 trades. Out of sample it made 9.6%.
The real search beat the luck search by 4.8% a year and 0.13 of Sharpe. That difference, not the headline backtest, is the size of the real effect.
There is a second sign. Watch the junk rules.
Junk conditions appeared in 7.2% of the rule pool. On no-edge data, the median top 25 still held 4.0% junk. On real SPY: 0.0%. Not one of the top 25 real rules used a junk condition.
Now the humbling part.
Of all 5,000 rules, 41.5% made money in sample. Of those, 84.5% still made money out of sample. But only 3.9% beat buy and hold in sample. Buy and hold made 6.9% a year in sample and 13.4% out of sample.
10 rules out of 5,000 beat buy and hold in both periods. That is 0.2%.
A real effect showed up. Beating the market with it was rare.
Can a coin flip match a trader?
On win rate, against one public rule, yes. In one SPY window, random entries won 61.25% of trades and a public RSI(2) rule won 60.34%. On return over worst drawdown, the rule led.
The AI validation study ran a public RSI(2) rule on SPY's last 10 years, June 2016 to June 2026, history it had held back. Next to it ran the seeded coin flip.
The coin won 61.25% of its trades. The rule won 60.34%. Rank them by win rate alone and the coin sits on top.
So never judge a rule by win rate alone. Pair it with a risk number.
Return over worst drawdown is the yearly return divided by the worst drop. Higher means more reward for the pain. The rule scored 0.0767. The coin scored 0.0494.
That is one window and one market. The coin is an average of 10 seeds, and the study gives no spread around it, so this is a measured gap, not a proven edge. The dollars are not a fair match either: the rule took 116 trades and the coin 93.2.
And buy and hold made $87,835.34 on $35,000. In the same study, 0 of 2,068 optimized strategies beat it out of sample.
Research on real traders points the same way. Barber, Lee, Liu and Odean studied day traders in The cross-section of speculator skill. They found a small group whose strong results kept going, and very few who profited reliably after fees. Fama and French did the same for fund managers in Luck versus Skill in the Cross-Section of Mutual Fund Returns. They compared real funds with simulated luck and found few that earned enough to cover their costs.
The comparison with chance comes first. Until a result has been measured against chance, it is still a story.
What does the skill actually consist of?
A process you can write down. The robustness study ran 11 checks on the first 75% of each market's history. Then it scored every variant on the untouched final 25%.
Across all 5,472 variants, 63.7% made money in the final quarter. 27.8% beat buy and hold there.
Passing at least 10 of the 11 checks is rare: 8.3% of variants did it. Of those variants, 98.0% made money in the untouched final quarter. Their median return was 7.8% a year, against 1.7% for all variants.
The strongest single check was the plainest one: still profitable in a slice of history held back for checking. Variants that passed it stayed profitable +67 points more often than those that failed. Next came profitable over the whole development period, +64 points, and still profitable after costs of 5 basis points a side, +60 points.
But the checks did not help with beating the market. 20.7% of the best variants beat buy and hold, against a 27.8% base rate. Staying profitable and beating the index are two different problems. The second is much harder.
Run the study's full gauntlet of seven filters and 262 of 5,472 variants survive. That is 4.8%.
Where the usual answers are right, and where they are wrong
Right: the method can be written down. Every check above is a rule you can run. None of them needs a gift.
Right: the comparison is a test, not a feeling. Mining the same way on no-edge data is how you see that luck's winners made a median -1.2% afterwards, while the best real rule made 9.6%.
Wrong: "anyone can succeed." Nothing here measures people, so nothing here backs that promise. It does show how rarely a strategy beat the market: only 27.8% of variants beat buy and hold in the final quarter.
Wrong: quoting a failure rate for traders. You will see one on many pages. I have no test behind any of those numbers, so I am not repeating one.
Is trading gambling?
It is gambling when you cannot say how your result differs from chance. A coin flip and a rule had nearly the same win rate. Only measuring them side by side showed the rule ahead on return over drawdown, in one window.
My working line: you are trading once you have measured your rule against chance before the money goes in. Skip that, and the market runs the test for you, with your money.
If you want a structured way to run these tests on your own strategies, it is what the Algo Trading Masterclass teaches.
The honest limits
- These studies test rules, not people. They show what a trader's skill has to beat. They do not measure any one trader.
- Long only, one unit, no leverage, no shorting, no stops, no position sizing. Adding any of those changes the numbers.
- No-edge data is a floor, not a model of markets. It shows how good a backtest can look by luck alone. It does not say real markets are random.
- The coin flip is one window and one market: SPY, June 2016 to June 2026, averaged over 10 seeds, with no stated spread.
- These markets, these dates. Every robustness result belongs to its 8 markets and their date ranges.
What this means for you
- Stop asking if you are talented. Ask what luck would have made. On no-edge data, the median best of 5,000 rules backtested at 6.4% a year.
- Distrust short records and long searches. 3 years and 384 tries gave luck a median Sharpe of 1.17.
- Pair win rate with a risk number. The coin won 61.25% of its trades, more than the rule. Return over drawdown put the rule ahead.
- Hold data back and test on it. Variants still profitable in the held-back slice stayed profitable in the final quarter +67 points more often than those that were not. That was the strongest single check.
- Always compare with buy and hold. 0 of 2,068 optimized strategies beat it out of sample.
Reference: the coin flip test in full
| SPY, June 2016 to June 2026 | Net profit | Return over worst drawdown | Win rate | Trades |
|---|---|---|---|---|
| Public RSI(2) rule | $22,568.94 | 0.0767 | 60.34% | 116 |
| Random entries (mean of 10 seeds) | $8,201.11 | 0.0494 | 61.25% | 93.2 |
| Buy and hold | $87,835.34 | 1 |
Starting capital $35,000, no compounding. Random entries use the same 5-bar hold as the rule.
Talent finds patterns. Skill checks them against the coin.
If you want the next study like this before anyone else, it goes out in the Overfit newsletter. StatOasis.com/Overfit
Methodology: no new backtest, so no execution rule applies to this article. Every measured number is quoted from two published StatOasis studies: Robustness testing (5,472 strategy variants on 8 markets, 5,000 random rules mined on SPY and on 10 zero-edge datasets) and AI strategy validation (SPY 1993 to 2026, out of sample from 2016, with a seeded random-entry control averaged over 10 seeds). The robustness study reads signals at the close and enters at the next open, long only, one unit. The AI study states its rules on its own page. The studies measure rules, not traders. Past results are not a promise about the future. Research, not investment advice.
