/skills-eval

name: skills-eval
description: 'Evaluate Claude skill quality through auditing. Use when reviewing or auditing skills.'
alwaysApply: false
category: skill-management
tags:
- evaluation
- improvement
- skills
- optimization
- quality-assurance
- tool-use
- performance-metrics
dependencies:
- modular-skills
tools: []
provides:
  infrastructure:
  - evaluation-framework
  - quality-assurance
  - improvement-planning
  patterns:
  - skill-analysis
  - token-optimization
  - modular-design
  sdk_features:
  - agent-sdk-compatibility
  - advanced-metrics
  - dynamic-discovery
estimated_tokens: 1800
usage_patterns:
- skill-audit
- quality-assessment
- improvement-planning
- skills-inventory
- tool-performance-evaluation
- dynamic-discovery-optimization
- advanced-tool-use-analysis
- programmatic-calling-efficiency
- context-preservation-quality
- token-efficiency-optimization
- modular-architecture-validation
- integration-testing
- compliance-reporting
- performance-benchmarking
complexity: advanced
evaluation_criteria:
  structure_compliance: 25
  metadata_quality: 20
  token_efficiency: 25
  tool_integration: 20
  claude_sdk_compliance: 10
modules:
- modules/advanced-tool-use-analysis.md
- modules/authoring-checklist.md
- modules/evaluation-criteria.md
- modules/evaluation-framework.md
- modules/evaluation-workflows.md
- modules/integration-testing.md
- modules/integration.md
- modules/performance-benchmarking.md
- modules/pressure-testing.md
- modules/trigger-isolation-analysis.md
- modules/troubleshooting.md
- modules/skill-authoring-best-practices.md
role: entrypoint

Skills Evaluation and Improvement

When NOT To Use

• Writing a new skill (use `abstract:skill-authoring`)
• Evaluating hooks (use `abstract:hooks-eval`)
• Evaluating rules (use `abstract:rules-eval`)

Overview

This framework audits Claude skills against quality standards to improve performance and reduce token consumption. Automated tools analyze skill structure, measure context usage, and identify specific technical improvements. Run verification commands after each audit to confirm fixes work correctly.

The scripts are in `plugins/abstract/scripts/`. `skills_auditor.py` provides structural analysis, while `improvement_suggester.py` builds a plan for one skill. `compliance_checker.py` reports standards findings. `token_usage_tracker.py` and `tool_performance_analyzer.py` measure tokens and tool run time.

Quick Start

Basic Audit

Run a full audit of all skills or target a specific file to identify structural issues.

# Audit all skills
make audit-all

# Audit specific skill
make audit-skill TARGET=path/to/skill/SKILL.md

Analysis and Optimization

Use `skill_analyzer.py` for complexity checks and `token_estimator.py` to verify the context budget.

make analyze-skill TARGET=path/to/skill/SKILL.md
make estimate-tokens TARGET=path/to/skill/SKILL.md

Improvements

Generate a prioritized plan and verify standards compliance using `improvement_suggester.py` and `compliance_checker.py`.

make improve-skill TARGET=path/to/skill/SKILL.md
make check-compliance TARGET=path/to/skill/SKILL.md

Evaluation Workflow

Start with `make audit-all` to inventory skills and identify high-priority targets. For each skill requiring attention, run analysis with `analyze-skill` to map complexity. Generate an improvement plan, apply fixes, and run `check-compliance` to verify the skill meets project standards. Finalize by checking the token budget for efficiency.

Evaluation and Optimization

Quality assessments use `skills_auditor.py` and `improvement_suggester.py` to generate reports. Performance analysis covers token use through `token_usage_tracker.py` and tool run time through `tool_performance_analyzer.py`. For standards compliance, `compliance_checker.py` reports findings and changes no files. Apply its fixes by hand.

Scoring and Prioritization

We evaluate skills across five dimensions: structure compliance, content quality, token efficiency, activation reliability, and tool integration. Scores above 90 represent production-ready skills, while scores below 50 indicate critical issues requiring immediate attention.

Improvements are prioritized by impact. Critical issues include security vulnerabilities or broken functionality. High-priority items cover structural flaws that hinder discoverability. Medium and low priorities focus on best practices and minor optimizations.

Structural Patterns

**Deprecated**: `skills/shared/modules/` directories. Shared modules must be relocated into the consuming skill's own `modules/` directory. The evaluator flags any remaining `skills/shared/` as a structural warning.

**Current**: Each skill owns its modules at `skills/<skill-name>/modules/`. Cross-skill references use relative paths (e.g., `../skill-authoring/modules/description-writing.md`).

Resources

Shared Modules: Cross-Skill Patterns

• **Skill Selection**: See [skill-selection-judgment.md](../../shared-modules/skill-selection-judgment.md)
• **Trigger Patterns**: See [trigger-patterns.md](modules/evaluation-criteria.md)

Skill-Specific Modules

• **Trigger Isolation Analysis**: See `modules/trigger-isolation-analysis.md`
• **Authoring Checklist**: See `modules/authoring-checklist.md`
• **Evaluation Workflows**: See `modules/evaluation-workflows.md`
• **Advanced Tool Use Analysis**: See `modules/advanced-tool-use-analysis.md`
• **Evaluation Framework**: See `modules/evaluation-framework.md`
• **Integration Patterns**: See `modules/integration.md`
• **Troubleshooting**: See `modules/troubleshooting.md`
• **Pressure Testing**: See `modules/pressure-testing.md`
• **Integration Testing**: See `modules/integration-testing.md`
• **Performance Benchmarking**: See `modules/performance-benchmarking.md`

Tools and Automation

• **Tools**: Executable analysis utilities in `scripts/` directory.

Exit Criteria

• [ ] Every audited skill receives a score across all five