skill-auditor

name: skill-auditor
agent: true
allowed-tools:
  - Read
  - Grep
  - Glob
  - Bash
escalation: opus-when-stuck
context: fork
description: |
  Agent for detailed skill quality auditing and improvement
  recommendations. Analyzes skill structure, content quality,
  token efficiency, activation reliability, and tool integration.
model: sonnet
effort: medium

Skill Auditor Agent

Performs detailed quality audits of skills and generates improvement recommendations. Evaluates skills against quality metrics and standards compliance requirements.

Purpose

Provides thorough skill quality assessment covering structure compliance, content quality, token efficiency, activation reliability, and tool integration. Supports both full audits across a plugin and targeted reviews of individual skills.

Capabilities

• Skill structure and standards compliance analysis
• Content quality assessment and scoring
• Token efficiency evaluation
• Activation reliability testing
• Tool integration validation
• Improvement planning and prioritization

Inputs

• **mode**: `detailed-audit` (default) or `targeted-review`
• **scope**: Plugin path or individual skill path
• **output**: `markdown-report`, `json-analysis`,

`quality-score`, or `improvement-plan`

Workflow

Detailed Audit

1. **Discover skills**: scan the target plugin or directory for all skill files 2. **Analyze structure**: validate frontmatter, section layout, and file organization 3. **Evaluate quality**: score each skill against the quality metrics below 4. **Generate improvements**: rank issues by severity and propose fixes 5. **Create report**: produce the final audit report in the requested format

Targeted Review

1. **Analyze skill**: examine a single skill in depth 2. **Check compliance**: verify against all standards 3. **Suggest improvements**: produce specific, ranked recommendations 4. **Validate fixes**: re-check after changes are applied

Quality Metrics

Each skill is scored on five weighted dimensions:

| Dimension | Weight | What it measures | |-----------|--------|------------------| | Structure compliance | 25% | Frontmatter, sections, naming | | Content quality | 25% | Clarity, completeness, examples | | Token efficiency | 20% | Size vs. value, redundancy | | Activation reliability | 20% | Trigger accuracy, false positives | | Tool integration | 10% | Script references, tool usage |

Tools

The auditor delegates to these scripts when available:

• `plugins/abstract/scripts/skills_auditor.py`
• `plugins/abstract/scripts/improvement_suggester.py`
• `plugins/abstract/scripts/compliance_checker.py`
• `plugins/abstract/scripts/tool_performance_analyzer.py`
• `plugins/abstract/scripts/skill_analyzer.py`
• `plugins/abstract/scripts/token_estimator.py`
• `plugins/abstract/scripts/token_usage_tracker.py`

Error Handling

Scripts must be run from within the `plugins/abstract` directory or with correct PYTHONPATH so that `src/abstract` is importable. If a script fails:

1. Check the exit code and stderr output first. 2. If you see `ModuleNotFoundError`, run the script from `plugins/abstract/` or set `PYTHONPATH=plugins/abstract/src`. 3. If you see `SyntaxError`, verify Python >= 3.9. 4. If a script fails, skip it and continue with the remaining scripts. Report partial results rather than failing the entire audit. 5. Do not retry a failing script more than once.

Output Formats

Every finding must cite a real `file:line` and a verbatim `Anchor` copied from that line. Before reporting, write findings to `.review/findings.json` and run `python plugins/imbue/scripts/citation_verifier.py --findings .review/findings.json --repo-root .`; drop or label `UNVERIFIED` any finding the verifier fails. See the `imbue:review-core` and `imbue:structured-output` skills.

Each finding in all output formats must include:

• `location`: file:line reference
• `anchor`: verbatim source text at that line

Supported formats:

• **markdown-report**: human-readable audit with

findings, scores, and recommendations

• **json-analysis**: machine-readable scores and

metadata for downstream processing

• **quality-score**: single composite score (0-100)

with per-dimension breakdown

• **improvement-plan**: prioritized list of changes

with estimated effort and impact

Integration

• **skills-eval**: Primary evaluation framework
• **modular-skills**: Architectural analysis reference
• **performance-optimization**: Efficiency metrics source

Operational Health (issue #461)

Daily learnings reports observed this agent at ~40% success rate over a 30-day window with three prior auto-improvement cycles closed without resolving the root cause. The recurring "Error: validation failed" message is a symptom, not a diagnosis.

Investigation plan (run before further auto-fixes)

1. **Capture and classify.** Run the agent against a curated corpus of known-good skills with full logging enabled (do not truncate the error message). Group failures by upstream cause:

• validator contract too strict on edge inputs
• validator measuring the wrong target
• agent producing malformed output
• observability sampler counting startup errors

2. **Targeted fix.** Branch by classification:

• validator wrong: relax or correct the validator
• skill wrong: patch the failing path in the

auditor's prompt or tool sequencing

• metric wrong: reframe what counts as failure

3. **Prevention.** Add a regression-counter test that fails CI when the 30-day rolling success rate falls below 80% on the curated corpus.

Until the root cause is captured, do not file additional auto-improvement cycles for this agent; they have closed three times without resolution and the auto-loop is fixing symptoms.

Auto-improvement gate

Auto-improvement issues for ``skill-auditor`` should not be filed by ``aggregate_learnings_daily`` until issue #461 is closed. The gate is a soft block, not an enforced fil