/skill-evaluation

name: skill-evaluation
description: >
  Evaluate any agent skill against a merged framework — Anthropic's Claude Code
  best practices plus Matt Pocock's writing-great-skills methodology — across
  4 axes (Trigger, Structure, Steering, Pruning). Produces an evidence-cited
  scorecard (0–100), a weighted overall score, and diagnosed failure modes
  with prioritized fixes. Use when the user asks to evaluate, rate, or audit
  a skill ("evaluate this skill", "skill scorecard", "review SKILL.md"), or
  to compare two skills.
metadata:
  author: ft.ia.br
  version: "2.1.0"
  date: 2026-07-03
  repository: https://github.com/fabricioctelles/skills
  license: Apache-2.0
  category: code-quality-and-review

Skill Evaluation

If you need the vocabulary and tests behind Axes 1, 3, and 4 (leading words, completion criteria, context pointers, the deletion test, failure-mode definitions), read `references/mechanics.md` before scoring those axes.

Source

• [Lessons from building Claude Code: How we use skills](https://claude.com/blog/lessons-from-building-claude-code-how-we-use-skills) — Anthropic, Jun 2026
• "The Missing Manual: How to Write Great Skills" — Matt Pocock, AI Engineer World's Fair 2026 ([video](https://www.youtube.com/watch?v=UNzCG3lw6O0)), and his `writing-great-skills` skill

Parameters

| Parameter | Description | Default | |-----------|-------------|---------| | `target` | Path to skill directory or SKILL.md to evaluate | Ask user | | `output` | Path to write the scorecard | `<target>/EVALUATION.md` | | `compare` | Optional second skill to compare side-by-side | None |

Also runs unattended: in CI, point `target` at skills changed in a PR and gate with `scripts/score.py --fail-below 60 ...` — non-zero exit below the threshold fails the check.

Criteria

18 criteria: 14 core, scored on every skill, plus 4 conditional criteria scored only when the skill's category makes them apply — otherwise mark **N/A** and exclude the criterion from both the numerator and denominator of the weighted average. Every score is 0–100 with evidence citing file, section, or line.

Axis 1 — Trigger (invocation)

| # | Criterion | Weight | Key question | |---|-----------|--------|---------------| | 1 | Invocation design | 2x | Is model-invoked vs. user-invoked deliberate and fitting? Model-invoked pays **context load** (the description loads every turn); user-invoked pays **cognitive load** (the human is the index). A skill that only ever fires by hand should be user-invoked. | | 2 | Description quality | 2x | Model-invoked: leading word up front, one trigger per branch (synonyms renaming the same branch are duplication), no identity that's redundant with the body. User-invoked (`disable-model-invocation: true`): a human-facing one-liner, no trigger list. Score against the mode the skill actually uses — never penalize a user-invoked skill for lacking trigger phrases. |

Axis 2 — Structure

| # | Criterion | Weight | Key question | |---|-----------|--------|---------------| | 3 | Steps vs. reference clarity | 1x | Does the skill distinguish ordered steps from on-demand reference? All-reference and all-steps skills are both valid — score clarity, not the mix. Is related material co-located (definition, rules, caveats under one heading)? | | 4 | Branch-aware disclosure & pointers | 2x | Is material every branch needs inline, and material only some branches need behind a context pointer? Does each pointer's wording say when to follow it ("if you need X, read Y")? A weakly worded pointer to must-have material is a variance bug. | | 5 | Conciseness (no sprawl) | 2x | Is SKILL.md lean — under 500 lines as a ceiling, smaller is better — with every line earning its context cost? | | 6 | Coherent scope | 1x | Does the skill do one thing and compose with others, rather than covering too much? |

Axis 3 — Steering

| # | Criterion | Weight | Key question | |---|-----------|--------|---------------| | 7 | Leading words | 2x | Does the skill use compact, high-prior terms ("vertical slice", "tight", "red") to anchor behavior, repeated consistently? Could any verbose passage collapse into one? | | 8 | Completion criteria & legwork | 2x | Skills with steps: does each step end on a checkable, exhaustive completion criterion? A vague one invites premature completion. Skills that are pure reference: is there an exhaustiveness bar over the reference itself ("every rule applied")? If neither applies, mark N/A. | | 9 | Gotchas section | 2x | Is there explicit capture of failure points, edge cases, footguns? | | 10 | Grounded in expertise | 2x | Does content come from observed failures and real project facts, or generic "best practices"? | | 11 | Avoids railroading | 1x | Does the skill leave room to adapt — procedures over declarations, defaults over menus — without over-prescribing? |

Axis 4 — Pruning

| # | Criterion | Weight | Key question | |---|-----------|--------|---------------| | 12 | No-ops (deletion test) | 2x | Running the deletion test sentence by sentence: if removing a sentence leaves behavior unchanged, it's a no-op — including restatements of what the model already does by default. Cite line numbers for candidates. | | 13 | Single source of truth | 1x | Does each meaning live in exactly one place? Duplication between SKILL.md and references/ counts too. | | 14 | Relevance & sediment | 1x | Are there stale lines, accumulated layers, or material that no longer influences what the skill does? |

Conditional criteria

Score only when the skill's category (from `references/categories.md`) makes the criterion apply; otherwise mark N/A and drop it from the weighted average entirely.

| # | Criterion | Weight | Applies to category | |---|-----------|--------|----------------------| | 15 | Setup flow | 1x | library-and-api-reference, data-fetching-and-analysis, ci-cd-and-deployment, infrastructure-operations | | 16 | Memory mechanism | 1x | business-process-automation, data-fetching