Skip to content
TrustedSkillsbeta
Back to all skills

Ab Test Framework Ml

Listed
dengineproblem·vlatest·testing

Helps with testing, machine learning as part of testing, debugging, and quality assurance workflows.

Repository

Install on your platform

This skill no longer installs under the name ab-test-framework-ml. dengineproblem/agents-monorepo is deleted or private, so it cannot be installed from.

We check each listing against its upstream repository with the rules the skills CLI uses, last on 2026-10-22. We don't show a command we know will fail.

Repository install instructions

What we know about this skill

Publisher

dengineproblem

Taken from the source listing. We haven't verified who controls this account.

Install target

Live repository

Not pinned. Installing fetches whatever the repository holds at the time you run the command.

Nobody has reviewed this code. TrustedSkills indexes and links skills; it does not audit, run or scan them. A skill runs with whatever access you give your agent — read the source before you install it.

Automated safety checks

scanned

We could not read this skill's source, so none of the checks below were run.

Not scanned. repository not found or private Source recorded as dengineproblem/agents-monorepo.
This is a static scan, not a review. It reads the skill's files; it does not run them, and it cannot tell you whether the skill is any good or whether the code does what its description says. A skill runs with whatever access you give your agent.
What each check looks for →

About this skill

What it does

This skill assists in designing, implementing, and analyzing A/B tests specifically for machine learning systems. It understands the unique challenges of testing ML models in production environments, including concept drift, model bias, statistical power calculations, and measuring both business metrics and model performance. The skill provides tools and guidance to ensure statistically sound experimentation and considers ML-specific factors like inference latency and model versioning.

When to use it

  • When designing an A/B test for a machine learning feature or model update.
  • To calculate the required sample size for an A/B test based on desired effect detection and statistical power.
  • For guidance on selecting appropriate metrics (both business and model performance) to track during an experiment.
  • When needing help with user randomization and traffic splitting within an A/B testing framework.

Key capabilities

  • Sample Size Calculation: Calculates the required sample size per variant based on baseline rate, minimum detectable effect (MDE), alpha, and power.
  • Traffic Splitting: Provides a mechanism for consistent user assignment to variants using hashing.
  • Guidance on Statistical Principles: Emphasizes the importance of defining primary and secondary metrics upfront, accounting for multiple testing corrections, and choosing appropriate randomization units.
  • ML-Specific Considerations: Addresses concerns like monitoring model performance metrics (accuracy, AUC), business metrics (conversion, revenue), inference latency, temporal effects, and model versioning.

Example prompts

  • "Calculate the sample size needed to detect a 5% relative improvement in a baseline conversion rate of 20%."
  • "How should I randomize users for an A/B test of my recommendation engine?"
  • "What metrics should I track when testing a new fraud detection model?"

Tips & gotchas

  • Requires understanding of basic statistical concepts like alpha, power, and effect size.
  • The provided sample size calculation is based on specific assumptions; ensure these are appropriate for your experiment.
  • Consistent hashing ensures users are consistently assigned to the same variant, which is crucial for reliable results.