/package-evaluator

name: package-evaluator
description: 'Evaluates Claude Code package quality across 6 dimensions for all 7 package types, producing scored audit reports. Triggers on: "evaluate package", "audit agent quality", "score this hook", "package audit", "skill quality check". NOT for LLM prompts, use prompt-lab.'
metadata:
  version: 1.3.1
  category: review
  tags: [quality, audit, scoring, frontmatter]
  difficulty: intermediate
  phase: review

Package Evaluator

Packages that do not activate on relevant queries waste the entire investment in writing them. A skill can have deep, well-structured content and still deliver zero value if its frontmatter description lacks the trigger phrases users actually type. An agent without a decision tree produces inconsistent results. A hook without a handler script is inert. Quality evaluation catches trigger gaps, missing sections, structural deficiencies, and shallow content before deployment — turning a package from a static document into a reliable tool.

Reference Files

| File | Contents | | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | `references/evaluation-rubric.md` | Detailed 1-5 scoring criteria per dimension, weight justifications, type-specific criteria, worked examples for calibration |

Audit Modes

Two modes, selected by input:

• **Quick Audit**: Evaluate a single package. Produces a full per-dimension scored report

with findings, severity classifications, and recommendations.

• **Full Audit**: Evaluate all packages in the repository. Produces a comparative ranking

table sorted by overall score, plus condensed per-package summaries. Optionally filtered to a single package type.

Mode Selection

| Input | Mode | | ------------------------------------------------------- | ---------------------------------------------- | | Path to a specific package directory or definition file | Quick Audit | | "all", "every package", no path specified | Full Audit | | "--type agents" or type filter | Full Audit filtered to one type | | Multiple specific paths | Quick Audit for each, then comparative summary |

---

Evaluation Dimensions

Six dimensions, each scored 1-5. Weighted sum determines overall percentage.

D1: Frontmatter Quality (20%)

Evaluates the YAML frontmatter block for completeness and discoverability.

**Signals:**

• `name` field present and non-empty
• `description` field present and non-empty
• Description length between 200-800 characters (sweet spot for keyword density without bloat)
• Description contains explicit trigger phrases users would type
• Description includes a "Use this skill when..." clause or equivalent
• Description is keyword-dense, not generic filler

**Scoring constraints:** A description under 100 characters caps this dimension at 2/5. A missing `name` or `description` field caps at 1/5.

D2: Trigger Coverage (18%)

Evaluates whether the package activates on the queries users actually type.

**Signals:**

• Synonym breadth — multiple phrasings for the same intent (e.g., "review", "audit",

"critique", "evaluate", "assess", "check")

• Implied contexts — situations where the package applies even without explicit keywords

(e.g., "user provides a design doc and asks for feedback")

• Domain-specific terms relevant to the package's function
• Explicit trigger phrase list in the description frontmatter
• Coverage of both imperative ("review this") and interrogative ("is this good?") forms

**Scoring constraints:** Fewer than 3 distinct trigger phrases caps at 2/5. Zero trigger phrases in the description caps at 1/5.

D3: Structural Completeness (20%)

Evaluates whether the package contains the sections needed to function reliably.

**Signals:**

• Prerequisites or setup instructions (if applicable)
• Multi-phase workflow or step-by-step procedure
• Error handling guidance or edge case documentation
• Output format specification (template, example, or schema)
• Limitations or scope boundaries stated
• Reference file table (if references/ directory exists)
• Calibration rules or quality gates

**Scoring constraints:** A package with no workflow section caps at 2/5. A package with a workflow but no error handling or output format caps at 3/5.

D4: Content Depth (22%)

Evaluates the substantive quality of the package's guidance — whether it provides enough detail for an agent to execute well without human intervention.

**Signals:**

• Multi-step workflows with decision points, not bare command lists
• Error cases documented with recovery actions
• Decision frameworks (when to do X vs Y, mode selection tables)
• Verbatim output examples or templates
• Severity classifications or scoring rubrics (where applicable)
• Cross-cutting analysis or synthesis steps beyond simple checklists

**Scoring constraints:** A package consisting only of bare commands with no explanatory context caps at 2/5. Reference files count toward this dimension only if they contain substantive guidance (checklists, rubrics, criteria), not just link collections.

D5: Consistency and Integrity (12%)

Evaluates internal consistency and structural integrity.

**Signals:**

• Directory name matches the `name` field in frontmatter exactly
• All files referenced in the definition file exist on disk (reference files, scripts, assets)
• Description content aligns with body content (description does not promise features

the body does not deliver)

• Consistent terminology throughout (same concept uses same term)

-