Index of every page on this site: llms.txt
All skills
wshobson avatar

/backtesting-frameworks

@4220308
by Seth Hobsonwshobson/agents43k stars
4,518
Run it yourself

Build robust backtesting systems for trading strategies with proper handling of look-ahead bias, survivorship bias, and transaction costs. Use when developing trading algorithms, validating strategies, or building backtesting infrastructure.

Use this Skill: https://skilld.dev/gh/wshobson/agents/backtesting-frameworks

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈66 tokens for metadata: the name and description. ≈805 when used: this file. ≈4.5k more on demand in 1 file.

Description uses 3.3% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈266 characters, or 3.3%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Backtesting Frameworks

Build robust, production-grade backtesting systems that avoid common pitfalls and produce reliable strategy performance estimates.

When to Use This Skill

  • Developing trading strategy backtests
  • Building backtesting infrastructure
  • Validating strategy performance
  • Avoiding common backtesting biases
  • Implementing walk-forward analysis
  • Comparing strategy alternatives

Core Concepts

1. Backtesting Biases

Bias Description Mitigation
Look-ahead Using future information Point-in-time data
Survivorship Only testing on survivors Use delisted securities
Overfitting Curve-fitting to history Out-of-sample testing
Selection Cherry-picking strategies Pre-registration
Transaction Ignoring trading costs Realistic cost models

2. Proper Backtest Structure

Historical Data
      │
      ▼
┌─────────────────────────────────────────┐
│              Training Set               │
│  (Strategy Development & Optimization)  │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│             Validation Set              │
│  (Parameter Selection, No Peeking)      │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│               Test Set                  │
│  (Final Performance Evaluation)         │
└─────────────────────────────────────────┘

3. Walk-Forward Analysis

Window 1: [Train──────][Test]
Window 2:     [Train──────][Test]
Window 3:         [Train──────][Test]
Window 4:             [Train──────][Test]
                                     ─────▶ Time

Detailed worked examples and patterns

Detailed sections (starting with ## Implementation Patterns) live in references/details.md. Read that file when the navigation summary above is insufficient.

Best Practices

Do's

  • Use point-in-time data - Avoid look-ahead bias
  • Include transaction costs - Realistic estimates
  • Test out-of-sample - Always reserve data
  • Use walk-forward - Not just train/test
  • Monte Carlo analysis - Understand uncertainty

Don'ts

  • Don't overfit - Limit parameters
  • Don't ignore survivorship - Include delisted
  • Don't use adjusted data carelessly - Understand adjustments
  • Don't optimize on full history - Reserve test set
  • Don't ignore capacity - Market impact matters

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts2mo5 checks · Risk SAFE
  • Gen Agent Trust Hub2mo

    The skill provides standard, clean framework templates and code implementations for financial backtesting. No security vulnerabilities or malicious patterns were detected.

  • Socket2mo

    No alerts

  • Snyk2mo

    Risk: LOW · No issues

  • Runlayer8mo

    1 file scanned · No issues

  • ZeroLeaks7mo

    2 findings · Score: 80/100

Signed by skilld at 4220308. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 days ago.

Activeupdated 6 months ago
  • Python
  • backtesting
  • trading
  • strategy
  • bias
  • walk-forward
  • out-of-sample
  • transaction-costs

README badge

README badge for wshobson/agents/backtesting-frameworks

Provides patterns and best practices for building backtesting systems that account for look-ahead bias, survivorship bias, transaction costs, and overfitting. Use this skill when developing trading strategy backtests, implementing walk-forward analysis, or validating strategy performance on out-of-sample data.

Generated from the current SKILL.md.

Does this skill cover specific backtesting libraries like Backtrader or VectorBT?
No. The skill focuses on backtesting principles and biases rather than implementations in particular libraries. It teaches the concepts you need to avoid look-ahead bias, survivorship bias, and overfitting regardless of your chosen framework.
What biases does this skill address?
It covers look-ahead bias, survivorship bias, overfitting, selection bias, and transaction costs. Each has a concrete mitigation strategy documented in the skill.
Does this include walk-forward analysis?
Yes. The skill includes walk-forward analysis patterns and explains how to structure rolling windows for strategy validation.
Can I use this for crypto or forex strategies?
The principles apply universally to any asset class with historical data, though the skill does not provide asset-class-specific implementations or examples.

Generated from the current SKILL.md. These answers refresh after source changes.