Backtest Expert

What it does

The Backtest Expert skill provides a systematic methodology for analyzing trading strategies against historical data, prioritizing robustness over simply maximizing paper profits. It guides users through a rigorous backtesting process that includes hypothesis definition, rule-based coding of strategy logic, stress testing parameters and execution conditions, and out-of-sample validation to assess real-world viability. The goal is to identify strategies that perform consistently well under pessimistic conditions, increasing the likelihood of success in live trading.

When to use it

• Developing or validating systematic trading strategies
• Evaluating a trading idea's robustness before live implementation
• Troubleshooting misleading backtest results
• Learning proper backtesting methodology and avoiding common pitfalls like curve-fitting and look-ahead bias
• Assessing how sensitive a strategy is to parameter changes and market conditions

Key capabilities

• Hypothesis Definition: Guides users in clearly articulating the trading edge.
• Rule-Based Coding: Enforces complete specificity and eliminates subjective judgment when defining entry, exit, position sizing, filters, and universe criteria.
• Stress Testing: Facilitates testing parameter sensitivity (stop loss, profit target, timing) and execution friction (slippage, order rejections).
• Time Robustness Analysis: Enables assessment of year-by-year performance and regime dependence.
• Sample Size Validation: Provides guidelines for ensuring sufficient trade volume for reliable results.
• Out-of-Sample Validation: Supports walk-forward analysis to optimize on one period and test on another.

Example prompts

• "Help me define a hypothesis for a trading strategy based on stocks gapping up on earnings."
• "What are the key considerations when stress testing my backtest for parameter sensitivity?"
• “Guide me through setting up a walk-forward analysis to validate this trading strategy.”

Tips & gotchas

• Clear Hypothesis is Key: The skill emphasizes defining your trading edge in a single, clear sentence before proceeding.
• Rule-Based Only: All aspects of the strategy (entry, exit, position sizing) must be defined with unambiguous rules – no subjective judgment allowed.
• Stress Testing is Crucial: Allocate most of your testing time to stress testing parameters and execution conditions.