Index of every page on this site: llms.txt
All skills
tradermonty avatar

/signal-postmortem

@f95e770
Run it yourself

Record and analyze post-trade outcomes for signals generated by edge pipeline and other skills. Track false positives, missed opportunities, and regime mismatches. Feed results back to edge-signal-aggregator weights and skill improvement backlog.

Use this Skill: https://skilld.dev/gh/tradermonty/claude-trading-skills/signal-postmortem

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈66 tokens for metadata: the name and description. ≈2.2k when used: this file. ≈2.8k more on demand in 2 files.

Description uses 3.3% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈266 characters, or 3.3%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Signal Postmortem

Overview

Signal Postmortem records and analyzes the outcomes of trading signals generated by the edge pipeline, screeners, and other skills. It compares predicted edge direction against 5-day and 20-day realized returns, categorizes outcomes (true positive, false positive, missed opportunity, regime mismatch), and generates feedback for edge-signal-aggregator weight adjustments and skill improvement backlog entries.

When to Use

  • After a trade has been closed and you want to record the outcome
  • When reviewing a batch of signals that have reached their holding period (5 or 20 days)
  • To identify systematic false positive patterns from specific skills
  • To generate feedback for edge-signal-aggregator weight calibration
  • When building a skill improvement backlog from decision quality metrics
  • For periodic (weekly/monthly) signal quality audits

Prerequisites

  • Python 3.9+
  • FMP API key (optional, for fetching realized returns if not provided manually)
  • Standard library + requests for API calls
  • Input: signal records in JSON format (from edge-signal-aggregator or screener outputs)

API Key Setup (Optional)

If you want to automatically fetch price data for return calculations, set up the FMP API key:

export FMP_API_KEY=your_api_key_here

Alternatively, pass the key via command line with --api-key YOUR_KEY. Without an API key, record manual outcomes from a saved signal with --signals-file, --signal-id, --exit-price and --exit-date. The saved signal must include a finite positive entry price.

Workflow

Step 1: Prepare Signal Records

Gather closed or matured signal records. Each record should include:

  • signal_id: Unique identifier
  • ticker: Stock symbol
  • signal_date: Date signal was generated
  • predicted_direction: LONG or SHORT
  • source_skill: Which skill generated the signal
  • entry_price: Price at signal generation (required for manual recording; batch may fetch it)
# Example: List signals ready for postmortem (5+ days old)
python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --list-ready \
  --signals-dir state/signals/ \
  --min-days 5

Step 2: Record Outcomes

Run the postmortem recorder to fetch realized returns and classify outcomes.

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signals-file state/signals/aggregated_signals_2026-03-10.json \
  --holding-periods 5,20 \
  --output-dir reports/

Automatic lookup uses FMP’s stable historical endpoint for the exact requested date range. Access failures are unavailable evidence; no legacy endpoint is tried. Manual recording remains offline, and requests is required only for automatic lookup.

Batch recording requires observed finite positive entry and 5-day exit prices. The exit uses the first usable price from day 5 through day 9 (calendar days), with the observed date recorded. Without a key or usable prices, signals are skipped instead of being classified as NEUTRAL. A real observed flat return is still NEUTRAL. Missing/non-finite secondary returns (such as 20d) are omitted; missing/non-finite 5d returns skip the signal. Include 5 in --holding-periods for batch classification; reordering periods does not change the primary exit.

The batch summary reports processed and skipped. A batch with zero processed signals exits 1 and writes only a diagnostic batch summary; mixed batches save valid records and exit 0. Check skipped counts before using the results. Skipped signals do not overwrite existing records. Previously saved records are not repaired or excluded automatically; review older records before analyzing them.

For manual outcome recording (when price data is already available):

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signals-file state/signals/aggregated_signals_2026-03-10.json \
  --signal-id sig_aapl_20260310_abc \
  --exit-price 178.50 \
  --exit-date 2026-03-15 \
  --outcome-notes "Closed at target, +3.2% in 5 days" \
  --output-dir reports/

Manual mode requires exactly one matching saved signal; it never infers metadata from the ID or fetches prices. Entry and exit must be finite positive prices, dates must be canonical YYYY-MM-DD, and exit cannot precede signal date. Missing/ambiguous source evidence or an unrepresentable return fails without writing or replacing an outcome. Source and output must be different files, including symlink/hardlink aliases. Supply the source file when using the manual example; an ID and exit alone are insufficient.

Manual classification uses (exit - entry) / entry, the saved LONG/SHORT direction and existing thresholds. The record carries return_basis: actual_exit and realized_returns.actual; holding_days is elapsed calendar days. This is raw price change, not position-sized, leveraged or fee-adjusted P&L. Actual exit returns do not populate the 5d/20d benchmark fields. The analyzer counts their outcomes and months but averages only observed finite 5d values, with return_samples_5d coverage and null/N/A when none exist. Existing fabricated records are not automatically repaired; review historical data separately.

Step 3: Classify Outcomes

The recorder automatically classifies each signal into one of four categories:

Category Definition
TRUE_POSITIVE Predicted direction matched realized return sign
FALSE_POSITIVE Predicted direction opposite to realized return
MISSED_OPPORTUNITY Signal not taken but would have been profitable
REGIME_MISMATCH Signal failed due to market regime change

Classification rules are documented in references/outcome-classification.md.

Step 4: Generate Feedback Files

Generate feedback for downstream consumers:

# Generate weight adjustment suggestions for edge-signal-aggregator
python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --generate-weight-feedback \
  --output-dir reports/

# Generate skill improvement backlog entries
python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --generate-improvement-backlog \
  --output-dir reports/

Step 5: Review Summary Statistics

Generate aggregate statistics by skill, by ticker, and by time period:

python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --summary \
  --group-by skill,month \
  --output-dir reports/

Output Format

Postmortem Record (JSON)

{
  "schema_version": "1.0",
  "postmortem_id": "pm_sig_aapl_20260310_abc",
  "signal_id": "sig_aapl_20260310_abc",
  "ticker": "AAPL",
  "signal_date": "2026-03-10",
  "source_skill": "edge-signal-aggregator",
  "predicted_direction": "LONG",
  "entry_price": 172.50,
  "realized_returns": {
    "5d": 0.032,
    "20d": 0.058
  },
  "exit_price": 178.50,
  "exit_date": "2026-03-15",
  "holding_days": 5,
  "outcome_category": "TRUE_POSITIVE",
  "regime_at_signal": "RISK_ON",
  "regime_at_exit": "RISK_ON",
  "outcome_notes": "Clean breakout, held through minor pullback",
  "recorded_at": "2026-03-17T10:30:00Z"
}

Weight Feedback (JSON)

{
  "schema_version": "1.0",
  "generated_at": "2026-03-17T10:35:00Z",
  "analysis_period": {
    "from": "2026-02-01",
    "to": "2026-03-15"
  },
  "skill_adjustments": [
    {
      "skill": "vcp-screener",
      "current_weight": 1.0,
      "suggested_weight": 0.85,
      "reason": "15% false positive rate in RISK_OFF regime",
      "sample_size": 42
    }
  ],
  "confidence": "MEDIUM",
  "min_sample_threshold": 20
}

Skill Improvement Backlog Entry (YAML)

- skill: vcp-screener
  issue_type: false_positive_cluster
  severity: medium
  evidence:
    false_positive_rate: 0.15
    sample_size: 42
    regime_correlation: RISK_OFF
  suggested_action: "Add regime filter or reduce signal confidence in RISK_OFF"
  generated_by: signal-postmortem
  generated_at: "2026-03-17T10:35:00Z"

Summary Report (Markdown)

Reports are saved to reports/ with filenames postmortem_summary_YYYY-MM-DD.md.

Resources

  • scripts/postmortem_recorder.py -- Records individual signal outcomes
  • scripts/postmortem_analyzer.py -- Generates feedback and summary statistics
  • references/outcome-classification.md -- Classification rules and edge cases
  • references/feedback-integration.md -- How to integrate feedback with downstream skills

Key Principles

  1. Honest Attribution -- Every outcome is attributed to its source skill for accountability
  2. Regime Awareness -- Regime context is recorded to distinguish skill failure from market regime shifts
  3. Minimum Sample Size -- Weight adjustments require 20+ signals for statistical validity
  4. Feedback Loop Closure -- Results flow back to improve both signal aggregation and skill quality

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts3d4 checks · Risk SAFE
  • Gen Agent Trust Hub4d

    The skill records and analyzes stock trading outcomes using a financial data API. It employs dynamic module loading for internal components and includes a minor surface for indirect prompt injection within its generated Markdown reports.

  • Socket3d

    No alerts

  • Snyk4d

    Risk: LOW · No issues

  • ZeroLeaks7mo

    Score: 93/100 · 2 sections analyzed

Signed by skilld at f95e770. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 5 days ago

README badge

README badge for tradermonty/claude-trading-skills/signal-postmortem