/pair-trade-screener

Cointegration Testing Guide

Table of Contents

1. What is Cointegration?
2. Cointegration vs Correlation
3. Augmented Dickey-Fuller (ADF) Test
4. Practical Implementation
5. Interpreting Results
6. Half-Life Estimation
7. Testing for Structural Breaks
8. Case Studies

What is Cointegration?

Intuitive Explanation

Imagine two drunk people walking home from a bar. They're both stumbling randomly, but one person has a dog on a leash. The person and the dog may wander in different directions temporarily, but the leash keeps them together in the long run. They are "cointegrated."

In finance:

• Person A = Stock A price
• Person B (with dog) = Stock B price
• Leash = Economic relationship (sector, supply chain, competition)

While both stock prices are non-stationary (random walks), their difference (or spread) is stationary because the economic "leash" pulls them back together.

Mathematical Definition

Two non-stationary time series X(t) and Y(t) are cointegrated if there exists a coefficient β such that:

Spread(t) = X(t) - β * Y(t)

is stationary (mean-reverting).

Key Components:

• X(t), Y(t): Non-stationary price series (random walks)
• β: Cointegration coefficient (hedge ratio)
• Spread(t): Stationary series (mean-reverting)

Why Cointegration Matters for Pair Trading

Without Cointegration:

• Prices can drift apart indefinitely
• No guarantee of mean reversion
• High risk of permanent losses

With Cointegration:

• Prices have long-term equilibrium
• Temporary deviations are predictable
• Mean reversion is statistically ensured

Example:

Non-Cointegrated Pair:

Stock A: Oil producer
Stock B: Tech company
Correlation: 0.75 (recent coincidence)

Result: No economic linkage → correlation breaks down → prices diverge forever

Cointegrated Pair:

Stock A: Exxon (XOM)
Stock B: Chevron (CVX)
Correlation: 0.92
Cointegration p-value: 0.008 (strong)

Result: Same sector, similar business → prices stay together → mean reversion reliable

Cointegration vs Correlation

Key Differences

Why Correlation Alone is Insufficient

Problem with Correlation:

Two random walks can have high correlation by chance without any fundamental relationship.

Example:

import numpy as np

# Generate two independent random walks
np.random.seed(42)
walk_A = np.cumsum(np.random.randn(252))
walk_B = np.cumsum(np.random.randn(252))

# Calculate correlation
correlation = np.corrcoef(walk_A, walk_B)[0, 1]
# Result: Might be 0.60-0.80 purely by chance!

Key Point:

• High correlation ≠ Mean reversion
• Need cointegration to ensure spread is stationary

Combining Correlation and Cointegration

Best Practice:

Use both as filters:

1. Correlation (≥ 0.70): Quick screen for co-movement
2. Cointegration (p < 0.05): Rigorous test for mean reversion

Decision Matrix:

Augmented Dickey-Fuller (ADF) Test

Purpose

The ADF test determines whether a time series has a unit root (non-stationary) or is stationary.

Hypotheses:

• Null (H0): Series has unit root (non-stationary)
• Alternative (H1): Series is stationary

For pair trading:

• Test the spread (not individual prices)
• Reject H0 → Spread is stationary → Pair is cointegrated

Test Procedure

Step 1: Calculate Spread

spread = price_A - (beta * price_B)

Step 2: Run ADF Test

from statsmodels.tsa.stattools import adfuller

result = adfuller(spread, maxlag=1, regression='c')
adf_statistic = result[0]
p_value = result[1]
critical_values = result[4]

Parameters:

• maxlag=1: Number of lags (typically 1 for daily data)
• regression='c': Include constant term (drift)
• Alternatives: 'ct' (constant + trend), 'n' (no constant)

Step 3: Interpret Results

if p_value < 0.05:
    print("Reject null → Spread is stationary → Cointegrated")
else:
    print("Fail to reject null → Not cointegrated")

ADF Test Equation

The ADF test estimates:

ΔSpread(t) = α + β*Spread(t-1) + Σ(γ_i * ΔSpread(t-i)) + ε(t)

Where:

• ΔSpread(t) = Spread(t) - Spread(t-1) (first difference)
• β: Coefficient of interest (tests for unit root)
• α: Drift term
• Σ(γ_i * ΔSpread(t-i)): Lagged differences (capture serial correlation)

Test Statistic:

ADF = β / SE(β)

Decision Rule:

• If ADF < Critical Value → Reject null (stationary)
• If p-value < 0.05 → Reject null (stationary)

Critical Values

Standard Critical Values (constant, no trend):

Example:

ADF Statistic: -3.75
Critical Value (5%): -2.86

Since -3.75 < -2.86 → Reject null → Stationary

Practical Implementation

Complete Python Example

import pandas as pd
import numpy as np
from statsmodels.tsa.stattools import adfuller
from scipy import stats

# Step 1: Load price data
prices_A = pd.Series([100, 102, 104, 103, 105, ...])  # Stock A
prices_B = pd.Series([50, 51, 52, 51.5, 52.5, ...])    # Stock B

# Step 2: Calculate beta (hedge ratio)
slope, intercept, r_value, p_value, std_err = stats.linregress(prices_B, prices_A)
beta = slope

print(f"Beta (hedge ratio): {beta:.4f}")
print(f"Correlation: {r_value:.4f}")

# Step 3: Calculate spread
spread = prices_A - (beta * prices_B)

# Step 4: Run ADF test
adf_result = adfuller(spread, maxlag=1, regression='c')

adf_statistic = adf_result[0]
p_value = adf_result[1]
critical_values = adf_result[4]
n_lags = adf_result[2]

# Step 5: Display results
print("\n=== Cointegration Test Results ===")
print(f"ADF Statistic: {adf_statistic:.4f}")
print(f"P-value: {p_value:.4f}")
print(f"Number of Lags: {n_lags}")
print(f"\nCritical Values:")
for key, value in critical_values.items():
    print(f"  {key}: {value:.4f}")

# Step 6: Interpret
if p_value < 0.01:
    print("\n✅ STRONG Cointegration (p < 0.01)")
    strength = "★★★"
elif p_value < 0.05:
    print("\n✅ MODERATE Cointegration (p < 0.05)")
    strength = "★★"
else:
    print("\n❌ NOT Cointegrated (p > 0.05)")
    strength = "☆"

# Step 7: Calculate half-life (if cointegrated)
if p_value < 0.05:
    from statsmodels.tsa.ar_model import AutoReg

    model = AutoReg(spread, lags=1)
    result = model.fit()
    phi = result.params[1]

    half_life = -np.log(2) / np.log(phi)
    print(f"\nHalf-Life: {half_life:.1f} days")

    if half_life < 30:
        print("  → Fast mean reversion (excellent)")
    elif half_life < 60:
        print("  → Moderate mean reversion (good)")
    else:
        print("  → Slow mean reversion (acceptable)")

FMP API Integration

import requests
import pandas as pd

def get_price_history(symbol, api_key, days=730):
    """Fetch historical prices from FMP API"""
    url = f"https://financialmodelingprep.com/api/v3/historical-price-full/{symbol}?apikey={api_key}"
    response = requests.get(url)
    data = response.json()

    # Extract historical prices
    hist = data['historical'][:days]
    hist = hist[::-1]  # Reverse to chronological order

    df = pd.DataFrame(hist)
    df['date'] = pd.to_datetime(df['date'])
    df = df.set_index('date')

    return df['adjClose']

# Example usage
api_key = "YOUR_API_KEY"
prices_AAPL = get_price_history("AAPL", api_key)
prices_MSFT = get_price_history("MSFT", api_key)

# Align dates
common_dates = prices_AAPL.index.intersection(prices_MSFT.index)
prices_AAPL = prices_AAPL.loc[common_dates]
prices_MSFT = prices_MSFT.loc[common_dates]

# Test for cointegration
slope, intercept, r_value, p_value, std_err = stats.linregress(prices_MSFT, prices_AAPL)
beta = slope
spread = prices_AAPL - (beta * prices_MSFT)

adf_result = adfuller(spread, maxlag=1)
print(f"AAPL/MSFT Cointegration p-value: {adf_result[1]:.4f}")

Interpreting Results

P-Value Interpretation

What p-value means:

• Probability of observing test statistic if null hypothesis (unit root) is true
• Lower p-value = stronger evidence against null = stronger cointegration

Guidelines:

ADF Statistic Interpretation

More negative = stronger cointegration:

ADF < -4.0: Very strong (★★★★)
ADF -3.5 to -4.0: Strong (★★★)
ADF -3.0 to -3.5: Moderate (★★)
ADF -2.5 to -3.0: Weak (★)
ADF > -2.5: Not cointegrated (☆)

Example Rankings:

Pair A: ADF = -4.25, p = 0.002 → ★★★★ (Best)
Pair B: ADF = -3.65, p = 0.018 → ★★★ (Excellent)
Pair C: ADF = -2.95, p = 0.042 → ★★ (Good)
Pair D: ADF = -2.45, p = 0.125 → ☆ (Reject)

Common Mistakes

Mistake 1: Testing Individual Prices

# WRONG: Testing if stock price is stationary
adf_result = adfuller(prices_A)  # ❌ Will always fail (prices are random walks)

Correct:

# RIGHT: Test if SPREAD is stationary
spread = prices_A - (beta * prices_B)
adf_result = adfuller(spread)  # ✅ Tests for cointegration

Mistake 2: Ignoring Lag Selection

# WRONG: Using too many lags (overfitting)
adf_result = adfuller(spread, maxlag=20)  # ❌ Too complex

# RIGHT: Use simple lag structure
adf_result = adfuller(spread, maxlag=1)  # ✅ Appropriate for daily data

Mistake 3: Confusing Correlation with Cointegration

# WRONG: Assuming high correlation = cointegration
if correlation > 0.80:
    trade_pair()  # ❌ Not sufficient

# RIGHT: Test for cointegration explicitly
if correlation > 0.70 and cointegration_pvalue < 0.05:
    trade_pair()  # ✅ Both conditions required

Half-Life Estimation

What is Half-Life?

Half-life measures how quickly the spread mean-reverts. Specifically, it's the expected time for the spread to move halfway back to its mean.

Example:

Current spread: +2.0 (2 std devs above mean)
Half-life: 20 days

Expected spread after 20 days: +1.0 (halfway to mean)
Expected spread after 40 days: +0.5 (halfway from +1.0 to 0)

AR(1) Model Approach

Model spread as autoregressive process:

S(t) = α + φ * S(t-1) + ε(t)

Where:

• φ: Autocorrelation coefficient (persistence)
• φ close to 1.0 → Slow mean reversion (long half-life)
• φ close to 0.0 → Fast mean reversion (short half-life)

Half-Life Formula:

Half-Life = -ln(2) / ln(φ)

Python Implementation

from statsmodels.tsa.ar_model import AutoReg

# Fit AR(1) model to spread
model = AutoReg(spread, lags=1)
result = model.fit()

# Extract autocorrelation coefficient
phi = result.params[1]

# Calculate half-life
half_life = -np.log(2) / np.log(phi)

print(f"Autocorrelation (φ): {phi:.4f}")
print(f"Half-Life: {half_life:.1f} days")

Interpreting Half-Life

Trading Implications:

Fast Half-Life (< 30 days):

• ✅ Quick profits
• ✅ Lower holding risk
• ✅ Frequent opportunities
• ❌ Transaction costs matter more

Slow Half-Life (> 60 days):

• ✅ More stable relationships
• ❌ Capital tied up longer
• ❌ Fewer trading opportunities
• ❌ Higher holding risk (regime changes)

Half-Life Stability

Test half-life over multiple periods:

# Calculate rolling half-life
rolling_half_life = []

for i in range(252, len(spread)):
    window = spread[i-252:i]
    model = AutoReg(window, lags=1)
    result = model.fit()
    phi = result.params[1]
    hl = -np.log(2) / np.log(phi)
    rolling_half_life.append(hl)

# Check stability
std_hl = np.std(rolling_half_life)
mean_hl = np.mean(rolling_half_life)
cv = std_hl / mean_hl  # Coefficient of variation

if cv < 0.30:
    print("Half-life is STABLE (good)")
else:
    print("Half-life is UNSTABLE (warning)")

Testing for Structural Breaks

Why Structural Breaks Matter

Definition:

• A structural break is a sudden, permanent change in the cointegration relationship
• Examples: Merger, spin-off, business model pivot, regulatory change

Impact on Pair Trading:

• Break in cointegration → Spread no longer mean-reverts
• Holding pair through break → Large losses
• Must detect breaks early and exit

Chow Test

Tests for known breakpoint (e.g., specific corporate event date):

from statsmodels.stats.diagnostic import breaks_cusumolsresid

# Fit OLS regression
from scipy import stats
slope, intercept = stats.linregress(prices_B, prices_A)[:2]
residuals = prices_A - (slope * prices_B + intercept)

# Test for structural breaks
stat, pvalue = breaks_cusumolsresid(residuals)

if pvalue < 0.05:
    print("⚠️ STRUCTURAL BREAK DETECTED")
else:
    print("✅ No structural break")

Rolling Cointegration

Monitor cointegration over time:

rolling_pvalues = []

for i in range(252, len(prices_A)):
    window_A = prices_A[i-252:i]
    window_B = prices_B[i-252:i]

    slope, intercept = stats.linregress(window_B, window_A)[:2]
    spread_window = window_A - (slope * window_B)

    adf_result = adfuller(spread_window, maxlag=1)
    pvalue = adf_result[1]
    rolling_pvalues.append(pvalue)

# Plot rolling p-values
import matplotlib.pyplot as plt
plt.plot(rolling_pvalues)
plt.axhline(y=0.05, color='r', linestyle='--', label='Significance threshold')
plt.ylabel('P-Value')
plt.xlabel('Time')
plt.title('Rolling Cointegration P-Value')
plt.legend()
plt.show()

Interpretation:

• P-value stays below 0.05 → Cointegration stable ✅
• P-value crosses above 0.05 → Cointegration breaking down ⚠️
• P-value remains above 0.10 → Relationship broken ❌

Early Warning System

Exit conditions based on cointegration degradation:

# Calculate 90-day rolling cointegration p-value
recent_pvalue = calculate_rolling_cointegration(prices_A[-90:], prices_B[-90:])

if recent_pvalue > 0.10:
    print("🚨 EXIT SIGNAL: Cointegration broken")
    exit_pair()
elif recent_pvalue > 0.05:
    print("⚠️ WARNING: Cointegration weakening")
    reduce_position()
else:
    print("✅ Cointegration healthy")

Case Studies

Case Study 1: XOM/CVX (Strong Cointegration)

Background:

• Exxon Mobil (XOM) and Chevron (CVX)
• Both: Large oil & gas companies
• Same sector, similar business models

Analysis:

# 2-year data (2023-2025)
correlation: 0.94
beta: 1.08
adf_statistic: -4.25
p_value: 0.0008
half_life: 28 days

Interpretation:

• ✅ Very strong cointegration (p < 0.01)
• ✅ High correlation (0.94)
• ✅ Fast mean reversion (28 days)
• ✅ Economic linkage (same sector)

Rating: ★★★★ (Excellent pair)

Trade Signal (Example):

Current Z-Score: +2.3 (XOM expensive relative to CVX)
→ SHORT XOM, LONG CVX
Entry: Z > +2.0
Exit: Z < 0.0
Stop: Z > +3.0

Case Study 2: JPM/BAC (Moderate Cointegration)

Background:

• JPMorgan Chase (JPM) and Bank of America (BAC)
• Both: Large banks
• Different focus areas (JPM more investment banking, BAC more retail)

Analysis:

correlation: 0.85
beta: 1.35
adf_statistic: -3.12
p_value: 0.031
half_life: 42 days

Interpretation:

• ✅ Moderate cointegration (p = 0.031)
• ✅ Good correlation (0.85)
• ✅ Acceptable mean reversion (42 days)
• ⚠️ Different business mix (less perfect linkage)

Rating: ★★★ (Good pair)

Case Study 3: AAPL/TSLA (No Cointegration)

Background:

• Apple (AAPL) and Tesla (TSLA)
• Both: High-growth tech stocks
• Different businesses (consumer electronics vs EVs)

Analysis:

correlation: 0.72
beta: 0.88
adf_statistic: -2.15
p_value: 0.182
half_life: N/A (not stationary)

Interpretation:

• ❌ No cointegration (p = 0.182)
• ✅ Moderate correlation (0.72)
• ❌ No mean reversion
• ❌ Weak economic linkage

Rating: ☆ (Reject pair)

Why correlation failed:

• Both had high returns in 2023-2024 (growth stock rally)
• Correlation driven by macro factors (interest rates), not fundamental linkage
• Likely to diverge when market conditions change

Case Study 4: Structural Break Example (GE)

Background:

• General Electric (GE) underwent major restructuring 2018-2021
• Spun off healthcare (GEHC) and energy divisions

Analysis:

# Pre-spinoff (2018-2020): GE/UTX pair
correlation: 0.81
p_value: 0.025 (cointegrated)

# Post-spinoff (2021-2023): GE/UTX pair
correlation: 0.52
p_value: 0.235 (NOT cointegrated)

Lesson:

• Corporate actions can break cointegration
• Must monitor for structural breaks
• Exit pairs when cointegration deteriorates

Summary Checklist

Before trading a pair, verify:

Statistical Checklist

• Correlation ≥ 0.70 (preferably ≥ 0.80)
• Cointegration p-value < 0.05 (preferably < 0.03)
• ADF statistic < -3.0
• Half-life 20-60 days
• No structural breaks in recent 6 months

Economic Checklist

• Same sector or supply chain relationship
• Similar business models
• No pending M&A or restructuring
• Similar market cap and liquidity
• Both stocks shortable (for short leg)

Risk Checklist

• Transaction costs < expected profit
• Adequate liquidity (>1M avg volume)
• Position sized appropriately (10-15% max)
• Stop loss defined (Z > ±3.0)
• Maximum holding period set (90 days)

Document Version: 1.0 Last Updated: 2025-11-08 References:

• Engle & Granger (1987): "Co-Integration and Error Correction"
• Hamilton (1994): "Time Series Analysis" (Chapter 19)
• Tsay (2010): "Analysis of Financial Time Series" (Chapter 8)