Regime-Based Investing

This tutorial shows how to detect market regimes, analyze their statistical properties, build a regime-conditional portfolio, and backtest it against buy-and-hold.

Financial markets alternate between distinct states – bull/bear, high/low volatility, risk-on/risk-off. Strategies that ignore these regime shifts suffer from unstable parameters and unexpected drawdowns. Regime detection lets you adapt.

Step 1: Detect Regimes with a Gaussian HMM

A Hidden Markov Model (HMM) assumes returns are generated by an unobserved Markov chain that switches between K states, each with its own mean and variance.

import wraquant as wq
import pandas as pd
import numpy as np
from wraquant.regimes import fit_gaussian_hmm, select_n_states

# Load daily returns
prices = pd.read_csv("spy_prices.csv", index_col=0, parse_dates=True)["Close"]
daily_returns = wq.returns(prices)

# Determine optimal number of states (typically 2 or 3)
state_selection = select_n_states(daily_returns, max_states=5)
print(f"BIC scores: {state_selection['bic_scores']}")
print(f"Optimal states: {state_selection['optimal_n']}")
# Lower BIC is better. Usually 2 or 3 states for equity indices.

# Fit 2-state HMM
hmm = fit_gaussian_hmm(daily_returns, n_states=2)
print(f"\nState 0: mean={hmm['means'][0]:.5f}, vol={hmm['variances'][0]**0.5:.4f}")
print(f"State 1: mean={hmm['means'][1]:.5f}, vol={hmm['variances'][1]**0.5:.4f}")

The state with higher mean and lower variance is typically the “bull” regime. The state with lower (or negative) mean and higher variance is the “bear” regime.

Step 2: Analyze the Transition Dynamics

The transition matrix reveals how persistent each regime is and how frequently switches occur.

from wraquant.regimes import regime_statistics, regime_transition_analysis

# Transition probabilities
trans = hmm['transition_matrix']
print(f"P(stay in state 0) = {trans[0, 0]:.4f}")
print(f"P(stay in state 1) = {trans[1, 1]:.4f}")
# High diagonal values (>0.95) mean regimes are persistent.

# Per-regime statistics
stats = regime_statistics(daily_returns, hmm['states'])
print("\nPer-regime performance:")
for regime, s in stats.items():
    print(f"  Regime {regime}: mean={s['mean']:.5f}, vol={s['vol']:.4f}, "
          f"sharpe={s['sharpe']:.3f}, obs={s['count']}")

# Transition analysis
transitions = regime_transition_analysis(hmm['states'])
print(f"\nAvg bull duration: {transitions['avg_duration'][0]:.1f} days")
print(f"Avg bear duration: {transitions['avg_duration'][1]:.1f} days")

Step 3: Rolling Regime Probabilities

Instead of a hard 0/1 classification, use the posterior probability of each state at each point in time. This is smoother and better for blending strategies.

from wraquant.regimes import rolling_regime_probability

# Posterior probabilities for each state at each time
probs = rolling_regime_probability(daily_returns, hmm)
print(f"Current P(bull): {probs['state_0'].iloc[-1]:.4f}")
print(f"Current P(bear): {probs['state_1'].iloc[-1]:.4f}")

# Use the bull probability as a continuous allocation signal
# High P(bull) -> full equity, low P(bull) -> reduce exposure

Step 4: Regime-Conditional Portfolio

Build a portfolio that changes its allocation based on the detected regime. Hold equities in bull regimes, shift to a defensive posture in bear regimes.

from wraquant.regimes import regime_aware_portfolio

# Define regime-specific allocations
allocations = {
    0: {"equity": 1.0, "cash": 0.0},   # Bull: fully invested
    1: {"equity": 0.3, "cash": 0.7},   # Bear: defensive
}

# Compute regime-conditional returns
portfolio = regime_aware_portfolio(
    returns=daily_returns,
    states=hmm['states'],
    allocations=allocations,
)

print(f"Regime portfolio total return: {portfolio['total_return']:.4f}")
print(f"Buy-and-hold total return:     {portfolio['buy_hold_return']:.4f}")
print(f"Regime portfolio Sharpe:        {portfolio['sharpe']:.4f}")
print(f"Buy-and-hold Sharpe:            {portfolio['buy_hold_sharpe']:.4f}")

Step 5: Backtest the Regime Strategy

Compare the regime-conditional strategy against buy-and-hold with proper backtesting metrics.

from wraquant.backtest import performance_summary, generate_tearsheet
from wraquant.risk import max_drawdown, sharpe_ratio

regime_returns = portfolio['strategy_returns']
benchmark_returns = daily_returns

# Side-by-side comparison
regime_perf = performance_summary(regime_returns)
bh_perf = performance_summary(benchmark_returns)

print(f"{'Metric':<25} {'Regime':>12} {'Buy&Hold':>12}")
print("-" * 50)
print(f"{'Sharpe Ratio':<25} {regime_perf['sharpe_ratio']:>12.4f} "
      f"{bh_perf['sharpe_ratio']:>12.4f}")
print(f"{'Max Drawdown':<25} {regime_perf['max_drawdown']:>12.4f} "
      f"{bh_perf['max_drawdown']:>12.4f}")
print(f"{'Annual Return':<25} {regime_perf['annual_return']:>12.4f} "
      f"{bh_perf['annual_return']:>12.4f}")
print(f"{'Annual Vol':<25} {regime_perf['annual_volatility']:>12.4f} "
      f"{bh_perf['annual_volatility']:>12.4f}")

# The regime strategy should show lower drawdowns and comparable or
# better risk-adjusted returns. It sacrifices some upside for
# significant downside protection.

Step 6: Alternative – Changepoint Detection

If you need real-time alerts for structural breaks rather than full regime classification, use Bayesian online changepoint detection.

from wraquant.regimes import online_changepoint

# Detect changepoints in the return series
cp = online_changepoint(daily_returns)
print(f"Detected {len(cp['changepoints'])} changepoints")

# Each changepoint has a date and run-length probability
for c in cp['changepoints'][:5]:
    print(f"  Date: {c['date']}, Probability: {c['probability']:.4f}")

# Changepoints mark the transitions between regimes.
# Use them to trigger portfolio rebalances.

Next Steps