Does the COT Report Actually Work? 4,889 Backtests on Larry Williams' COT Index

Picking better trades is not the same as earning more for the drawdown you take. On this grid the filter did the first in 92.6% of variants and never beat the baseline's CAR/MaxDD.

Methodology & risk note: Backtested event study of the E-mini S&P 500 (@ES.D, day session) daily data, 2006-2025, 4,889 strategy variants, frictionless. Results are hypothetical and not investment advice. Past patterns don't guarantee future results. Full method and disclaimer below.

TL;DR

• The filter lifts the average trade in most variants. It never lifts CAR/MaxDD, the return you get for the percentage drawdown you sit through. 92.6% of the 4,888 filter-on variants beat the unfiltered baseline's $586.67 average trade (their median average trade is $796.56). Zero of them beat the baseline's 0.51 CAR/MaxDD. Their median CAR/MaxDD is 0.07.
• It never makes the account bigger than the unfiltered one either. Zero of the 4,888 beat the baseline's $238,187.50 net profit. Across the 4,888, median net profit is $76,675.00 and median trade count is 95 (two separate medians), against the baseline's 406.
• The featured setup trails the baseline on percentage drawdown and CAR/MaxDD. Filter on Small Speculators, avoid the 35 to 75 middle of their range, 24-week lookback: 158 trades, $159,212.50 net, a 2.63 profit factor, a 26.74% max drawdown against the baseline's 15.41%, and a CAR/MaxDD of 0.20 against 0.51. It was chosen after the fact, on the full sample, as the cell whose 27-cell neighbourhood has the highest mean CAR/MaxDD (0.199) inside Small Speculators, avoid-the-middle.
• The "smart money" wasn't the smart pick, and neither was anyone else. A Small Speculators bull extreme was followed by the strongest median 20-day return of the six group and side rows (1.302%), ahead of a Commercials bull extreme (1.081%) and the unconditional day (1.076%). It beat both the unconditional day and a Commercials bull extreme at three of four horizons (5, 10 and 20 days), on 62 observations, and no Small Speculators filter beat the unfiltered baseline's CAR/MaxDD.
• Verified twice, then timed honestly. An independent Python rebuild matched the original TradeStation grid on all 4,889 variants. The published grid then moves each report to the day it became public.

AvgTrade is the average dollar profit per trade. ProfitFactor is gross profit divided by gross loss. CAR/MaxDD is compound annual return divided by the worst drawdown: how much return you got per unit of pain.

Every week, you're handed the "smart money's" hand, and told to copy it

The US government publishes who is long and who is short S&P 500 futures once a week. The CFTC's Commitments of Traders report, the weekly public positioning file, normally comes out on the Friday of the week its positions are dated. It breaks the futures market into three groups: the commercial hedgers who actually use the underlying, the large speculators (hedge funds and money managers), and the small speculators, meaning the public. Larry Williams built a career reading it, turned it into a 0-to-100 index, and left the trading world one sticky rule: watch the Commercials, because they're the smart money.

That rule gets repeated in every forum and course, and almost nobody tests it. I repeated the follow-the-Commercials line myself for years before I ever put a number on it. So we did, with one rule held fixed from the first bar to the last: a report counts only from the close of its release day, never from the earlier date the data file stamps it with.

Two comfortable beliefs didn't survive contact with 4,889 backtests. The "smart money" Commercials, at a bull extreme, were followed by a median 20-day return of 1.081%, next to the 1.076% of an ordinary day. And the bigger belief, that bolting positioning data onto a strategy makes it a better strategy, failed in every single variant on the measure that pays for a drawdown.

We took the same plain RSI(2) mean-reversion rule at the heart of our RSI deep dive on trading the S&P 500 with mean reversion, bolted the COT index onto it exactly as Williams designed it, and ran 4,889 backtests. The questions were whether the filter earns its keep, and, separately, whether the folklore about which trader group to follow survives the numbers.

It raises the quality of the trades it keeps. It does not earn its keep.

How we tested: 4,889 backtests, and why the base strategy is deliberately boring

Here's exactly what sits behind every number below.

• Instrument: @ES.D, the E-mini S&P 500 futures contract, day session, daily bars.
• Window: January 3, 2006 through August 29, 2025 (4,959 daily bars).
• Base strategy: RSI(2) mean-reversion, long only. Buy the next bar at market when RSI(2) drops below 25 and the account is flat. Exit the next bar at market when RSI(2) climbs above 70. No filter, no discretion: the plainest swing-trade rule there is, so any lift or drag gets pinned on the COT filter and nothing else. It's the same base oscillator we ran through a 33,792-variant shootout in RSI vs Stochastic vs Williams %R. Here it's the fixed control, not the thing under test.
• The filter: Larry Williams' COT index, computed per trader group (Commercials, Non-Commercials, or Small Speculators) over a lookback window in weeks. The index scores where a group's current net position sits between its own highest and lowest net position over that lookback: 100 means the most bullishly positioned in the window, 0 the most bearishly positioned. Same math as a Stochastic oscillator, applied to positioning instead of price. Three filter modes gate entries: only trade below a lower level (fade the bearish extreme), only trade above an upper level (fade the bullish extreme), or avoid the whole middle band and trade only at either extreme.
• Report timing: each report counts from the close of its CFTC release day. That is the Friday of the week its positions are dated, or the next business day after a federal holiday in that week, with 35 dated exceptions. 1,026 reports land on a later bar than the date the data file stamps them with.
• The grid: one unfiltered baseline plus 4,888 filter-on combinations sweeping trader group x lookback (20 to 36 weeks, in 2-week steps) x filter mode x lower level x upper level: 4,889 variants total.
• Capital and costs: $35,000 starting capital, one ES contract per trade, one position at a time, no compounding, frictionless: no commission or slippage modeled, so the filter is judged on its own merits, not on execution costs.
• Reliability floor: 50 trades. 40 of the 4,889 variants fall below it, the smallest at 44 trades. They are flagged, not dropped, and none of them is the baseline or the featured variant.

The index value only refreshes on the days a group's net position actually changes, under the frozen interpretation used here. Every other day it holds a "not ready" value and blocks new entries. That's a structural reason the filter looks very different from the baseline in trade count, and it matters for how you read every table that follows.

Can we trust these numbers? Two independent engines, 4,889 for 4,889

Before any finding below is worth reading, it has to survive a harder test than "did the backtest run." Every trade count and net profit in the replication grid was produced twice, by two independent engines: one in TradeStation and one in Python, run on the same underlying data and reconciled line by line.

The baseline canary (the simplest possible check, filter off) is where verification starts. Our independent rerun produced 406 trades and $238,187.50 in net profit. The original TradeStation run produced 406 trades and $238,188.00. That's a match within a dollar, on a strategy that ran for more than 19 years.

Then the full grid. 4,889 out of 4,889 variants matched on trade count exactly, and on net profit within a dollar. Not a sample of the grid. Every single row.

That match shows the Python rebuild reproduces TradeStation, on the dates the data file stamps each report with. It does not check the timing, because a stamped date is earlier than the day the report became public. So the published grid takes one more step: every report moves to its release bar, and the numbers in the rest of this page come from that run. The unfiltered baseline reads no COT value at all, so it is identical in both runs.

This matters because a study is only as trustworthy as its rerun, and a rerun is only as trustworthy as its clock. It's the same standard we hold every study to, laid out in full in why most traders fail: the missing piece called robustness testing.

Finding 1: Does the COT filter actually make the strategy better?

Short answer: per trade, usually. On CAR/MaxDD, never. First the per-trade case, because it's the one the sales pages quote.

92.6% of the 4,888 filter-on variants beat the baseline's $586.67 average trade. The median filter variant makes $796.56 per trade. Profit factor points the same way: 80.9% of filter variants beat the baseline's 1.84, with a median of 2.11.

Now read the bottom four rows. Each is a separate median across the 4,888 filter variants, not the figures of one single variant. The median trade count is 95 against 406, the median net profit $76,675.00 against $238,187.50, and the median max drawdown 37.19% against 15.41%. The median CAR/MaxDD is 0.07 against 0.51. Zero of the 4,888 beat the baseline on that measure. The highest any of them reached was 0.36.

The median filter variant does beat one named random control. That control fires as often as the median of the 4,849 variants at or above the 50-trade floor and holds as long as their median, and it averaged a CAR/MaxDD of 0.012 (sd 0.012) over 10 seeds. The median filter variant's 0.07 sits far outside that spread. So on CAR/MaxDD the median filter variant lands above this seeded control and below simply taking every RSI(2) signal.

Finding 2: The featured setup, and why it was chosen after the fact

The featured cell in the grid is filtering on Small Speculators, avoiding the middle of their 0 to 100 range (trading only when the index sits below 35 or above 75), with a 24-week lookback.

Average trade rises from $586.67 to $1,007.67 and profit factor from 1.84 to 2.63. Net profit, win rate, percentage drawdown and CAR/MaxDD go the other way. Win rate slips from 75.86% to 74.05%. Net profit falls to $159,212.50. Max drawdown deepens from 15.41% to 26.74%, and CAR/MaxDD falls from 0.51 to 0.20.

Here is how the cell was picked, and what that pick is worth. It isn't "the best row in the grid." Inside Small Speculators, avoid-the-middle, we averaged CAR/MaxDD across every setting one grid step away in every direction (lookback ±2 weeks, lower level ±5, upper level ±5). This cell had the highest neighbourhood average, 0.199 over 27 cells. 18 single cells in the same 725-variant region score higher than it on their own, and the best of them reaches 0.23.

That search read the whole 2006 to 2025 sample. No years were held back to test the pick on, so the featured cell is chosen after the fact. Read it as the cell whose 27-cell neighbourhood has the highest mean CAR/MaxDD in that 725-variant Small Speculators, avoid-the-middle region, not as a result that has passed an out-of-sample test. A pick made after the fact, from the most flattering neighbourhood in its region, still trails the unfiltered strategy by more than half on CAR/MaxDD. The grid-wide count says the same thing: none of the 4,888 beats 0.51.

The wider bucket (Small Speculators, avoid-the-middle mode) holds 725 variants with a median CAR/MaxDD of 0.11. If you take one comparison away from this section, make it 0.20 next to 0.51: the featured cell against the strategy with no filter at all.

Finding 3: Which trader group actually works: Commercials, Non-Commercials, or Small Speculators?

This is where the study breaks from the textbook. Classic COT theory says follow the Commercials: they're the "smart money," so a Commercials bull extreme (positioning at its most bullish in the lookback window) should lead a stronger move than usual. The Small Speculators are supposed to be the opposite: the crowd, wrong at the turns, worth fading rather than following.

We tested that directly. Using the classic Williams extreme levels (75 and above for bull, 25 and below for bear) at the featured 24-week lookback, here's the median forward return, event versus the unconditional baseline, at four horizons:

Read the Commercials bull row first. It sits above the unconditional median at 1 day and 20 days, both narrowly (0.054% vs 0.049%, 1.081% vs 1.076%), and below it at 5 and 10 days (0.092% vs 0.281%, 0.268% vs 0.539%). The "follow the smart money" case doesn't show up here.

Small Speculators bull is the row that leans the other way. It runs above the unconditional median at 5 days (0.423% vs 0.281%), 10 days (0.781% vs 0.539%) and 20 days (1.302% vs 1.076%), and below it at 1 day (-0.017% vs 0.049%). Its 20-day median is the highest of all six rows. Small Speculators bear is the only row that holds one side of the unconditional line at every horizon, and it is the side below. Treat all of this as directionally suggestive at 59 to 87 events a row, not as a standalone signal. One observation in the Small Speculators bull row, dated 2025-08-29, sits on the last bar of the data and has no forward window, which is why that row reads n=62.

Worth knowing before you set an alert for these: extremes are a several-times-a-year event, not a daily one. Across the 2006 to 2025 window, each group hit a bull or bear extreme between 3.0 and 4.4 times per year per side (411 extreme-crossing events in total across all six group and side combinations). Once a group entered an extreme, mean run duration ranged from 3.6 to 5.2 consecutive weekly reports. So a COT extreme is less like a signal firing and more like a regime arriving. It shows up a few times a year and then holds for about a month.

Now run each group through the RSI(2) filter in the "avoid the middle" mode, the same mode that produced the featured variant.

Small Speculators have the highest median net profit, average trade, profit factor and CAR/MaxDD of the three groups. Non-Commercials have the shallowest median drawdown, 33.81% against 34.96%. Commercials, the group the textbook says to follow, have the lowest median profit factor and CAR/MaxDD and the deepest median drawdown, 43.70%. And the leader's 0.11 median CAR/MaxDD is still less than a quarter of the unfiltered baseline's 0.51. Winning the group contest does not mean beating no filter at all.

Across the whole grid, the single best CAR/MaxDD belongs to neither Small Speculators nor Commercials. It is a Non-Commercials cell in the same mode, at 0.36, on 191 trades with a 13.57% max drawdown. Selectivity is no quality guarantee either. Commercials fade-the-low-extreme trades a median of 77.5 times, the second-fewest of the nine group and mode buckets, and carries the lowest median average trade ($707.24) and profit factor (1.90) and the deepest median drawdown (51.04%) of all nine.

The Small Speculators and the Commercials also move in opposite directions almost by construction: their COT-index values carry a -0.4647 correlation over 1,016 comparable bars, since COT positioning is largely a zero-sum ledger across groups. When one group leans bullish, another tends to lean bearish, which is part of why "which group do you filter on" changes the answer this much.

None of this is causal proof that small traders are secretly smart. It's a pattern in one 20-year ES dataset that leans against the "just do what the big money does" instinct. That is the same instinct behind the 40-in, 20-out hedge fund trend strategy, a rule tested there as mechanical entries and exits, not as a bet on the institutions.

The honest catch: better trades, a smaller account than no filter, and a lower CAR/MaxDD

Here's the part we're not going to spin away, because a study that buries its own catch isn't worth the pixels.

The filter lifts the average trade in most variants. It does not make the strategy richer than the unfiltered one, and it does not lift its CAR/MaxDD. Of the 4,888 filter-on variants tested, exactly zero beat the baseline's total net profit of $238,187.50, and exactly zero beat its 0.51 CAR/MaxDD. Not the featured variant. Not any of the other 4,887. The median filter variant takes 95 trades across more than 19 years, against the baseline's 406.

One fair objection deserves its own paragraph. Drawdown percent is the largest dollar drop divided by the account's peak just before it, and a one-contract account that has grown less shows a bigger percentage on a smaller dollar loss. In dollars, the featured variant's worst drawdown is $31,700.00 against the baseline's $43,875.00, and 96.7% of filter variants have a smaller dollar drawdown than the baseline. Measured as net profit over that dollar drawdown, the featured variant still trails, 5.02 against 5.43. On that measure 165 of the 4,888 variants do beat the baseline, 3.4% of the grid. The percentage view is the house standard and the one CAR/MaxDD uses. The dollar view narrows the gap without closing it for the featured cell.

The trade count falls for a structural reason stated up top: the COT report is a weekly signal, and under the frozen interpretation used here, the index value only updates on the days a group's net position actually changes. Every other day, the gate carries no new information and blocks entry. Stack a slow, weekly-refresh gate onto a fast, 2-to-3-day RSI(2) swing trade, and the gate sits closed most of the time. The featured variant spends 15.37% of bars in a trade against the baseline's 44.59%. Better quality per trade, at the cost of trade count (no variant keeps more than 204 of the baseline's 406 trades), and the quality gain never outruns the lost volume in any of the 4,888 tries.

Larry Williams himself made a version of this point: the index isn't a light switch you flip on and off, and he never traded the level in isolation. His own approach stacks the COT reading with price and other conditions rather than trading it as a standalone trigger. This study only tested the stacked version on one strategy, and on that strategy the stack raised the average trade in 92.6% of variants and lowered CAR/MaxDD in all of them.

The verdict, and the honest limits

Does the COT report work? As a filter that raises the average trade on this RSI(2) long strategy on ES, yes, in 92.6% of 4,888 tested combinations. As a way to make more money than not filtering at all, or a higher CAR/MaxDD, no, not once, on this grid.

The "follow the smart money" folklore doesn't survive contact with the data either. The Commercials bull extreme was followed by a 20-day median barely different from an ordinary day, and within the avoid-the-middle filter mode the Commercials had the lowest median CAR/MaxDD of the three groups. The Small Speculators did best within that mode, and their best was still a fraction of the unfiltered strategy's.

Now the limits, because a study that hides them isn't worth trusting:

• The featured variant was chosen after the fact. Its selection read the full 2006 to 2025 sample, and no period was held back to test it. It is the cell whose 27-cell neighbourhood has the highest mean CAR/MaxDD inside Small Speculators, avoid-the-middle. It is not an out-of-sample result, and it is not the single optimal setting to run.
• Frictionless. Every number here is computed with no commission and no slippage. Net profit and average trade are reported before commission and slippage, and no cost model was run to size how much costs would reduce them.
• Long-only. The base RSI(2) strategy and every COT filter variant were tested on long entries only. Nothing here says anything about the short side.
• Single market. These results are @ES.D, the E-mini S&P 500 day session, and nothing else. They are not shown to generalize to other futures markets, other instruments, or other strategies.
• Report timing is a rule plus 35 dated exceptions. 34 of those exceptions are sourced to a CFTC schedule or press release. One, for the 2018-12-04 office closure, is an unverified bound. The largest delay past a release is 1 bar.
• Weekly data on a fast strategy. The COT report updates once a week, and its index only refreshes on days a group's net position changes. Most days, the gate carries no new information at all.
• Two drawdown views. The percentage drawdown that CAR/MaxDD uses reads deeper on an account that has grown less. In dollars most filter variants have a smaller worst drawdown, and 165 of 4,888 beat the baseline on net profit over dollar drawdown.
• Every number is from an independent rerun, verified 4,889 of 4,889 against the original TradeStation study on the data file's own dates, then run on release dates. None of it is sourced from the original export directly.

What this means for you

1. Judge a filter by what it does to the account, not to the average trade. On the one strategy tested here, an RSI(2) long on ES, stacking Williams' COT index raised the average trade in 92.6% of variants and lowered CAR/MaxDD in all of them. A per-trade number on its own tells you whatever you were hoping to hear.
2. Check the clock on every data point your backtest reads. A COT report is not public on the date the data file stamps it with. Read from that stamp, the cell the same neighbourhood rule picked scored a CAR/MaxDD of 0.68, better than the baseline's 0.51. Counted from release days, the pick moves to a different cell and scores 0.20.
3. Question the "follow the smart money" default. In this data the Commercials bull extreme was followed by a 20-day median next to an ordinary day's, and Commercials had the lowest median CAR/MaxDD of the three groups in the avoid-the-middle mode. Test your own assumption about which group matters before you filter on it.
4. Expect a weekly-refresh gate on a 2-to-3-day swing trade to sit closed most of the time. Under the frozen interpretation used here, the COT index refreshes only on the days a group's net position changes, and the median filtered variant takes 95 trades against the baseline's 406. This study ran one fast base strategy and did not compare it against a slower one.
5. A neighbourhood pick is still a pick made after the fact. Choosing the cell whose neighbourhood has the highest average CAR/MaxDD protects you from one lucky cell. It does not test whether the whole neighbourhood beats the unfiltered strategy. Here it did not, on net profit or CAR/MaxDD.

Get the next study in your inbox

This is one study in an ongoing series. If you want the next one, the same kind of large-scale, independently-verified, frictionless-but-honest backtest, join The Overfit newsletter at StatOasis.com/Overfit. It's where we publish the data behind the trading ideas everyone argues about.

The COT report does have something to tell you. On this strategy, it said which trades looked better. It never found a CAR/MaxDD better than no filter at all.

Disclaimer

All results above are derived from historical backtesting using daily @ES.D (E-mini S&P 500) price data and do not represent actual trading results. Backtested performance is hypothetical. Past performance of any pattern does not guarantee future results. This page is for educational and informational purposes only and does not constitute investment advice. StatOasis is not a registered investment advisor. Nothing here is a recommendation to buy or sell any security or futures contract. Please consult a licensed financial professional before making any investment decision.

Transparency: StatOasis sells trading-education products. Our research is produced independently and is not altered to favor a sale.

Freshness: The underlying data covers 2006 through August 2025. We re-review this study when the dataset is extended.

StatOasis. ES (E-mini S&P 500) day session, RSI(2) + Williams COT filter, 2006-2025. Backtested, frictionless, long-only, single market. Not financial advice.