Backtest Expert

What it does

The Backtest Expert skill provides a systematic methodology for analyzing trading strategies using historical data. It focuses on identifying robust strategies that perform consistently under stressful conditions, rather than simply maximizing paper profits. The tool helps users evaluate strategy viability, troubleshoot misleading backtests, and avoid common pitfalls like curve-fitting and look-ahead bias by incorporating friction and stress testing assumptions.

When to use it

• Developing or validating systematic trading strategies.
• Evaluating whether a trading idea is robust enough for live implementation.
• Troubleshooting why a backtest might be misleading.
• Learning proper backtesting methodology.
• Assessing parameter sensitivity and regime dependence.

Key capabilities

• Hypothesis Definition: Requires clear articulation of the trading edge being tested.
• Rule-Based Codification: Enforces strict, unambiguous rules for entry, exit, position sizing, filters, and universe selection.
• Stress Testing: Includes parameter sensitivity analysis (stop loss, profit target, timing), execution friction simulation (slippage, order rejection), and time robustness checks (year-by-year performance).
• Realistic Cost Modeling: Incorporates commissions and slippage estimates into backtest calculations.
• Sample Size Requirements: Provides guidelines for minimum trade counts to ensure reliable results.

Example prompts

• "Analyze this trading strategy: Stocks that gap up >3% on earnings and pull back to the previous day's close within the first hour provide a mean-reversion opportunity."
• "Stress test my strategy by varying the stop loss at 50%, 75%, 100%, 125%, and 150% of its baseline value."
• “Evaluate this backtest for potential look-ahead bias.”

Tips & gotchas

• Prerequisites: Requires Python 3.9 or higher to run the evaluation script.
• User-Provided Data: The skill requires users to provide historical data; it does not have any external data dependencies.
• Focus on Robustness: The primary goal is to find strategies that survive stressful conditions, not necessarily those with the highest potential profit.