/evaluate-skill

Evaluate Skill Execution

Manually evaluate a recent skill execution for qualitative feedback.

Usage

/abstract:evaluate-skill <skill-name>              # Evaluate most recent execution
/abstract:evaluate-skill <skill-name> --all        # Evaluate all recent executions
/abstract:evaluate-skill <skill-name> --date YYYY-MM-DD  # Evaluate specific date

Purpose

Captures human-in-loop qualitative feedback about skill effectiveness to improve the self-improvement loop. This data feeds into:

• Log aggregation (Phase 3)
• LEARNINGS.md pattern detection
• `/improve-skills` prioritization (Phase 5)

What It Does

1. **Loads recent executions** from `~/.claude/skills/logs/<plugin>/<skill>/` 2. **Presents execution summary** (timestamp, duration, outcome, context preview) 3. **Prompts for evaluation**:

• Effectiveness rating (1-5)
• Friction points encountered
• Improvement suggestions

4. **Updates log entry** with evaluation metadata 5. **Tracks evaluation completion** (prevents duplicate evaluations)

Evaluation Questions

1. Effectiveness Rating (1-5)

• **5 - Exceptional**: Skill exceeded expectations, no friction
• **4 - Effective**: Skill achieved goal with minor friction
• **3 - Adequate**: Skill worked but had noticeable inefficiencies
• **2 - Poor**: Skill partially failed or required significant workarounds
• **1 - Failed**: Skill did not achieve intended outcome

2. Friction Points

What slowed you down or caused issues?

• Skill missing key information?
• Instructions unclear or incomplete?
• Wrong tool recommendations?
• Excessive token usage?
• Outdated information?
• Missing error handling?

3. Improvement Suggestions

What would make this skill better?

• Add examples?
• Simplify language?
• Update workflow steps?
• Add error recovery guidance?
• Split into smaller skills?
• Add context-specific variations?

Output

Updates the log entry with:

{
  "qualitative_evaluation": {
    "evaluated_at": "2026-01-08T04:00:00Z",
    "rating": 4,
    "friction_points": ["Missing error handling for X", "Y was unclear"],
    "improvement_suggestions": ["Add example for Z", "Simplify step 3"],
    "evaluator_notes": "Worked well but needed clarification on..."
  }
}

Examples

Evaluate Recent Execution

/abstract:evaluate-skill proof-of-work

# Output:
# Found 3 executions of imbue:proof-of-work in the last 7 days
#
# Most recent execution:
# - Timestamp: 2026-01-08 03:15:23 UTC
# - Duration: 2.3s
# - Outcome: success
# - Context: Validated POW for PR #84
#
# Rate effectiveness (1-5): _

Evaluate All Recent

/abstract:evaluate-skill proof-of-work --all

# Evaluates all unevaluated executions from the last 7 days

Evaluate Specific Date

/abstract:evaluate-skill proof-of-work --date 2026-01-07

# Evaluates executions from 2026-01-07

Integration

**Phase 3** (Log Aggregation):

• Aggregator reads `qualitative_evaluation` field
• Calculates average ratings by skill
• Identifies common friction points
• Generates improvement recommendations

**Phase 5** (Self-Improvement):

• `/improve-skills` prioritizes low-rated skills
• Improvement suggestions guide skill updates
• Version tracking shows rating improvements over time

Related

• `Skill(abstract:skill-execution-logger)` - Captures initial execution data
• `/abstract:aggregate-logs` - Rolls up evaluations to LEARNINGS.md (Phase 3)
• `/abstract:improve-skills` - Acts on evaluation insights (Phase 5)

Hooks

None (manual invocation only).

Permissions

• **allowed-tools**: `["Read", "Write", "Grep"]`
• **permission-mode**: `"default"`

Version

1.0.0 (Phase 2 implementation)