Llm Debugger

What it does

The llm-debugger skill provides tools to inspect and understand the internal workings of Large Language Models (LLMs). It allows you to view intermediate states, trace token probabilities, and analyze the decision-making process within an LLM. This helps in diagnosing issues, improving prompt engineering, and gaining insights into model behavior.

When to use it

• Debugging unexpected outputs: When an LLM produces incorrect or undesirable results, use this skill to examine its internal state and identify potential causes.
• Prompt optimization: Understand how different prompts influence the LLM's token selection process for more effective prompt design.
• Model behavior analysis: Gain a deeper understanding of how an LLM arrives at its conclusions by observing its intermediate steps.
• Educational purposes: Learn about the inner workings of LLMs and how they generate text.

Key capabilities

• Token probability tracing
• Intermediate state inspection
• Decision-making process analysis
• Debugging tools for LLMs

Example prompts

• "Debug this prompt: [prompt text]"
• "Show me the token probabilities for the last 5 tokens generated."
• "Analyze the decision-making process when generating the response to 'Write a short story about a cat.'"

Tips & gotchas

The skill requires familiarity with LLM concepts like tokens and probability distributions. Results can be verbose, so focus on specific areas of interest for efficient debugging.