Mutation Testing

What it does

This skill automatically performs mutation testing on Metabase code, identifying potential data model vulnerabilities and improving reliability. It generates coverage reports, uses Claude to write tests, verifies mutations, and creates draft pull requests with associated Linear issues. The process is end-to-end, designed specifically for the Metabase development environment.

When to use it

• To proactively identify weaknesses in Metabase's data model logic.
• When introducing significant changes or refactoring existing code within Metabase.
• As part of a continuous integration/continuous delivery (CI/CD) pipeline for Metabase development.
• To improve the overall robustness and test coverage of Metabase features.

Key capabilities

• Generates mutation testing reports.
• Groups functions based on coverage relationships.
• Utilizes Claude to automatically generate tests.
• Creates Linear issues and draft pull requests for identified vulnerabilities.
• Supports specifying a base branch for comparisons.
• Allows associating issues with an existing Linear project.

Example prompts

• /mutation-testing metabase.lib.order-by - Runs mutation testing on the metabase.lib.order-by namespace.
• /mutation-testing metabase.lib.order-by --base-branch release-x.52.x - Runs mutation testing with a specified base branch.
• /mutation-testing metabase.lib.order-by --project-id abc-123 - Runs mutation testing and associates issues with the Linear project abc-123.

Tips & gotchas

• Requires a running nREPL connected to the Metabase development environment.
• You must have a valid LINEAR_API_KEY environment variable set, gh CLI authenticated with GitHub, and the claude CLI available on PATH.
• For the first invocation or after a REPL restart, you'll need to configure the Linear team ID using mut-test/set-config!.