backtesting-frameworks by wshobson

Backtesting Frameworks

Build robust, production-grade backtesting systems that avoid common pitfalls and produce reliable strategy performance estimates.

When to Use This Skill

Developing trading strategy backtests
Building backtesting infrastructure
Validating strategy performance
Avoiding common backtesting biases
Implementing walk-forward analysis
Comparing strategy alternatives

Core Concepts

1. Backtesting Biases

Bias	Description	Mitigation
Look-ahead	Using future information	Point-in-time data
Survivorship	Only testing on survivors	Use delisted securities
Overfitting	Curve-fitting to history	Out-of-sample testing
Selection	Cherry-picking strategies	Pre-registration
Transaction	Ignoring trading costs	Realistic cost models

2. Proper Backtest Structure

Historical Data
      │
      ▼
┌─────────────────────────────────────────┐
│              Training Set               │
│  (Strategy Development & Optimization)  │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│             Validation Set              │
│  (Parameter Selection, No Peeking)      │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│               Test Set                  │
│  (Final Performance Evaluation)         │
└─────────────────────────────────────────┘

3. Walk-Forward Analysis

Window 1: [Train──────][Test]
Window 2:     [Train──────][Test]
Window 3:         [Train──────][Test]
Window 4:             [Train──────][Test]
                                     ─────▶ Time

Detailed worked examples and patterns

Detailed sections (starting with ## Implementation Patterns) live in references/details.md. Read that file when the navigation summary above is insufficient.

Best Practices

Do's

Use point-in-time data - Avoid look-ahead bias
Include transaction costs - Realistic estimates
Test out-of-sample - Always reserve data
Use walk-forward - Not just train/test
Monte Carlo analysis - Understand uncertainty

Don'ts

Don't overfit - Limit parameters
Don't ignore survivorship - Include delisted
Don't use adjusted data carelessly - Understand adjustments
Don't optimize on full history - Reserve test set
Don't ignore capacity - Market impact matters

Third-party checks

No alerts16d5 checks · Risk SAFE

Gen Agent Trust Hub16d
The skill provides standard, clean framework templates and code implementations for financial backtesting. No security vulnerabilities or malicious patterns were detected.
Socket16d
No alerts
Snyk16d
Risk: LOW · No issues
Runlayer6mo
1 file scanned · No issues
ZeroLeaks5mo
2 findings · Score: 80/100

What it does

Provides patterns and best practices for building backtesting systems that account for look-ahead bias, survivorship bias, transaction costs, and overfitting. Use this skill when developing trading strategy backtests, implementing walk-forward analysis, or validating strategy performance on out-of-sample data.

Generated from the current SKILL.md.

Frequently asked

Does this skill cover specific backtesting libraries like Backtrader or VectorBT?

No. The skill focuses on backtesting principles and biases rather than implementations in particular libraries. It teaches the concepts you need to avoid look-ahead bias, survivorship bias, and overfitting regardless of your chosen framework.

What biases does this skill address?

It covers look-ahead bias, survivorship bias, overfitting, selection bias, and transaction costs. Each has a concrete mitigation strategy documented in the skill.

Does this include walk-forward analysis?

Yes. The skill includes walk-forward analysis patterns and explains how to structure rolling windows for strategy validation.

Can I use this for crypto or forex strategies?

The principles apply universally to any asset class with historical data, though the skill does not provide asset-class-specific implementations or examples.

Generated from the current SKILL.md. These answers refresh after source changes.

/backtesting-frameworks

SKILL.md