/check-skill

description: Generate and run a coder-eval activation suite for a Claude Code skill. Use when the user asks whether a skill triggers, wants to test skill activation, or worries a skill has silently stopped firing.
allowed-tools: ["Read", "Glob", "Grep", "Write", "Bash"]

Skill activation check

A skill only earns its keep if the model reaches for it at the right moment. That decision is made almost entirely from the skill's frontmatter `description` — so "does my skill trigger?" is a measurable question, and this is how you measure it: build a labelled set of user requests, run a real agent against each one, and score whether the skill was engaged.

The user's request is: `$ARGUMENTS`

Step 1 — Locate the target skill

`$ARGUMENTS` may be a path to a `SKILL.md`, a path to the skill's **directory**, a skill name, or empty.

• Empty: glob `.claude/skills/*/SKILL.md` and `**/skills/*/SKILL.md`. One match →

use it. Several → list them and ask which. None → say so and stop.

• A directory: use the `SKILL.md` inside it.
• A name: find the matching skill directory.

The **bare skill name is the directory name** containing `SKILL.md`. Read the frontmatter `description` and keep it in front of you: that string is what the model matches against, so it is the primary input to row design and the thing you will end up recommending edits to.

**Measure its length while you are there — two separate budgets truncate it, and either one produces a low-recall result that looks exactly like bad wording.**

1. **Per-skill truncation.** `description` and `when_to_use` are concatenated and cut at a fixed character budget — **1,536 characters**, configurable via the `skillListingMaxDescChars` setting. Trigger text past the cutoff cannot affect activation at all, so it may as well not exist. 2. **The whole-listing budget, which matters more in exactly the repositories that run activation suites.** The listing always contains every skill *name*, but its total character budget scales at about **1% of the model's context window**, shared across **every** skill the user has installed. When it overflows, Claude Code drops descriptions **starting with the skills you invoke least**.

The second one has a consequence worth stating plainly: in a many-skill repository a skill can score near-zero recall with a **perfectly good description**, because its description was never in the listing. Rewriting the wording then fixes nothing. And the drop order is least-invoked-first, so a *newly authored* skill — which is by definition rarely invoked, and is exactly what someone runs this suite on — is the most likely victim. That is a systematic bias against the skill under test.

Levers, if the listing is the problem: `skillListingBudgetFraction` (the 1% default), the `SLASH_COMMAND_TOOL_CHAR_BUDGET` environment variable (a fixed character count), and `skillOverrides` set to `"name-only"` to free budget from skills you do not need matched.

If the skill has **no `description`** in its frontmatter, stop and report that as the finding — a skill with no description can never be model-invoked, so a suite would score zero recall by construction and tell you nothing you don't already know.

Step 2 — Confirm coder-eval is installed

Run `coder-eval --version`. Installing this plugin did not install the CLI, and the suite cannot be validated or run without it.

If it is missing, follow `${CLAUDE_PLUGIN_ROOT}/reference/cli-setup.md`: offer the install, **ask before running it**, and confirm with `coder-eval --version` afterwards. Never install unprompted, and do not continue if the user declines.

That reference also covers the other half of the version check — whether this project pins a coder-eval version, and what to do when the installed one does not match it.

Step 3 — Check what already covers this skill

Before designing anything, look at what the repository already has. Locate the task tree per `${CLAUDE_PLUGIN_ROOT}/reference/repo-layout.md` and glob it for a task carrying a `skill_triggered` criterion.

**If an existing suite already names this skill**, report its coverage — how many positive and distractor rows, which `suite_thresholds` it gates on — and **offer to extend it** by appending rows to its dataset, rather than scaffolding a parallel suite. Two suites for one skill drift apart and the user pays for both on every run. Scaffold only when nothing covers this skill, or when the user chooses to after seeing what is there.

Two things worth reporting rather than silently working around:

• **An existing suite with no distractor rows.** Precision is 1.0 by definition, so half

its result is meaningless. Offer to add distractors — a real improvement over both scaffolding a rival suite and saying nothing.

• **A neighbouring suite that names a *different* skill.** That is not coverage. Go ahead

and scaffold, and say why the neighbour did not count.

**Then check where `agent.plugins` should come from.** If an experiment the task will resolve against already supplies that block, **inherit it and do not write one** — a task that redeclares what the experiment provides drifts from it, with nothing to catch the divergence. Write the template's own block only when nothing already exposes the skill to the sandbox. If several experiments exist and it is unclear which one this task resolves against, ask rather than guess.

Step 4 — Design the rows

This step is the whole experiment. The rest is mechanics.

**Positive rows** — requests a real user would plausibly make that the description claims to cover. Paraphrase; never copy phrasing out of the description. A row lifted from the description tests string matching, not activation. Vary the vocabulary and include at least one oblique row where the user describes their *problem* rather than the operation the skill performs.

**Distractor rows** — adjacent requests the skill should *not* claim, especially ones that share vocabulary with the