Evaluate and Optimize Agent Skills¶

Systematically measure and improve your Agent Skills using structured evals. You'll write test cases with expected outcomes, benchmark pass rates and token usage, compare skill versions with blind evaluation, and optimize skill descriptions for triggering accuracy — learning to refine skills with data, not intuition.

Objectives¶

• Evaluate skill quality by writing test cases with expected outcomes, running evals in isolated contexts, and benchmarking pass rates and token usage — completing at least one full test-measure cycle that produces quantitative performance data.
• Compare skill versions using blind A/B evaluation to determine whether changes actually improved performance, making refinement decisions based on evidence rather than intuition.
• Optimize skill descriptions for triggering accuracy by analyzing activation patterns against test prompts, reducing false positives (overly broad triggers) and false negatives (missed activations) to improve when the skill fires.

Back to course overview