Vet

What it does

The vet skill provides a mechanism for evaluating and assessing the quality of text generated by AI agents. It can analyze text based on criteria such as factual accuracy, clarity, and relevance to a given prompt. The skill aims to improve the reliability and trustworthiness of AI-generated content through automated evaluation.

When to use it

• Content Quality Assurance: Before publishing articles or reports drafted by an AI agent, use vet to ensure accuracy and readability.
• Prompt Refinement: Evaluate responses from an AI agent to identify areas where the prompt needs adjustment for better results.
• Training Data Validation: Assess the quality of text used to train AI models, identifying potential biases or inaccuracies.
• Automated Feedback Loops: Integrate vet into a workflow to automatically provide feedback to AI agents on their generated outputs.

Key capabilities

• Text evaluation based on defined criteria.
• Factual accuracy assessment.
• Clarity and readability analysis.
• Relevance scoring relative to the input prompt.

Example prompts

• "Vet this paragraph for factual errors: [paragraph text]"
• "Assess the clarity of this response: [AI agent's response]"
• "How relevant is this summary to the original article? [summary text] - Original Article: [article text]"

Tips & gotchas

The effectiveness of vet depends on clearly defining the evaluation criteria. Providing detailed instructions or a rubric will improve the quality of the assessment.