/cot-contrarian-detector

COT Index Calculation

Formula

COT Index = (current_net - window_min) / (window_max - window_min) * 100

Where current_net is the most recent week's large-speculator net position (non-commercial long minus non-commercial short), and window_min / window_max are the minimum and maximum net position observed over the lookback window (including the current week).

• 100 = current net position is the highest (most net-long) of the lookback window
• 0 = current net position is the lowest (most net-short) of the lookback window
• 50 = current net position sits at the midpoint of the window's range

This is the standard "Stochastics of positioning" formulation used across COT index literature (crowdedmarketreport.com, various CFTC-data commentary sites) — it is mathematically identical to a %K stochastic applied to net position instead of price.

Undefined cases: the index is None (not zero, not fifty) when:

• Fewer than lookback_weeks of history are available — there is nothing meaningful to compare "current" against
• window_max == window_min — every week in the window had the identical net position, so the range is zero and the division is undefined

Both cases are treated identically by cot_index.compute_cot_index(): return None rather than raising, so callers can decide how to surface "no data" (this skill's screener puts such markets in a skipped list with a reason, never silently drops them or reports a misleading 50).

Lookback Windows

This skill computes two indices per market:

• Primary: 156 weeks (~3 years), default --lookback-weeks. The conventional COT-index lookback — long enough to span multiple full positioning cycles (a market can stay "crowded" for months at a time), short enough to stay relevant to the current volatility/rate regime. This is the index used for the CROWDED_LONG/CROWDED_SHORT classification.
• Short: 26 weeks (~6 months), default --short-lookback-weeks. Context only — shown alongside the 3-year index so the user can tell whether an extreme is fresh (also extreme on the 26-week window) or aging (was more extreme weeks ago and has already started to normalize). A market that is CROWDED_LONG on both windows is a stronger, more current signal than one that is CROWDED_LONG only on the 3-year window because it peaked months ago.

Both lookbacks are CLI flags (--lookback-weeks, --short-lookback-weeks) so the user can adjust for a specific market's cycle length if needed.

Extreme Thresholds

Default thresholds: high >= 90, low <= 10 (--threshold-high, --threshold-low). These are inclusive boundaries — a market at exactly 90.0 classifies as CROWDED_LONG.

Sensitivity note: 90/10 is a conservative, "clearly extreme" threshold that will flag fewer markets but with higher conviction. Loosening to 80/20 roughly doubles the number of markets flagged in a typical universe scan but increases false positives — markets that are "elevated" rather than "crowded." There is no universally correct threshold; 90/10 is the conventional starting point in COT-index commentary and matches crowdedmarketreport.com's public framework. Tighten toward 95/5 for higher-conviction, fewer signals; loosen toward 80/20 for an earlier, noisier read.

Open Interest Normalization

compute_oi_normalized_net() expresses net position as a fraction of total open interest (net_position / openInterestAll). This exists because raw net position isn't comparable across markets of very different sizes — a net position of -50,000 contracts is trivial in a 2,000,000-contract-OI market (ES) but enormous in a 100,000-contract-OI market. Normalizing by OI gives a rough cross-market comparability check; it is not used in the CROWDED_LONG/SHORT classification itself (which is purely about the market's own historical range via the COT Index), but is surfaced in reports as additional context.

Returns None when open interest is zero, missing, or non-numeric — a division by an unreliable denominator should never silently produce a misleading ratio.

Legacy vs. Disaggregated Report

The CFTC publishes COT data in two formats:

• Legacy report (used by this skill): three trader categories — Non-Commercial (large speculators), Commercial (hedgers), and Non-Reportable (small traders below reporting thresholds). This is the original, longest-running COT format and matches Shapiro's own framework of fading the speculative crowd.
• Disaggregated report (not used): splits traders into four finer categories (Producer/Merchant, Swap Dealers, Managed Money, Other Reportables) for physical-commodity futures only (not financials). More granular but not available for financial futures (equity indices, rates, FX) and not what Shapiro's methodology is built around.

This skill exclusively uses the legacy report's non-commercial long/short fields (noncommPositionsLongAll / noncommPositionsShortAll) as the "large speculator" proxy across all 65 markets FMP's COT API covers, so the same methodology applies uniformly to indices, rates, FX, metals, energy, agri, and crypto futures.

FMP API Field Glossary

Field names as returned by stable/commitment-of-traders-report (verified live against the FMP API, 2026-07). All fields below are on a single weekly report row, one row per market per Tuesday-dated report.

Handling Data Gaps

sort_dedupe_rows() sorts rows ascending by date and, when two rows share the same date (observed occasionally with re-published/corrected reports), keeps the last one in the input order — treated as the more recently fetched/corrected value. Rows missing a date are dropped rather than kept in an arbitrary position, since an undated row can't be placed in the lookback window correctly.