/hooks-eval

name: hooks-eval
description: 'Evaluate hook security, performance, and SDK compliance. Use for audits.'
alwaysApply: false
category: hook-management
tags:
- hooks
- evaluation
- security
- performance
- claude-sdk
- agent-sdk
dependencies:
- hook-scope-guide
provides:
  infrastructure:
  - hook-evaluation
  - security-scanning
  - performance-analysis
  patterns:
  - hook-auditing
  - sdk-integration
  - compliance-checking
  sdk_features:
  - python-sdk-hooks
  - hook-callbacks
  - hook-matchers
estimated_tokens: 1200
modules:
- modules/evaluation-criteria.md
- modules/sdk-hook-types.md
role: entrypoint

When NOT To Use

• Writing a new hook (use `abstract:hook-authoring`)
• Evaluating skills (use `abstract:skills-eval`)
• Evaluating rules in `.claude/rules/` (use `abstract:rules-eval`)

Hooks Evaluation Framework

Overview

This skill provides a detailed framework for evaluating, auditing, and implementing Claude Code hooks across all scopes (plugin, project, global) and both JSON-based and programmatic (Python SDK) hooks.

Key Capabilities

• **Security Analysis**: Vulnerability scanning, dangerous pattern detection, injection prevention
• **Performance Analysis**: Execution time benchmarking, resource usage, optimization
• **Compliance Checking**: Structure validation, documentation requirements, best practices
• **SDK Integration**: Python SDK hook types, callbacks, matchers, and patterns

Core Components

| Component | Purpose | |-----------|---------| | **Hook Types Reference** | Complete SDK hook event types and signatures | | **Evaluation Criteria** | Scoring system and quality gates | | **Security Patterns** | Common vulnerabilities and mitigations | | **Performance Benchmarks** | Thresholds and optimization guidance |

Quick Reference

Hook Event Types

HookEvent = Literal[
    "PreToolUse",  # Before tool execution
    "PostToolUse",  # After tool execution
    "UserPromptSubmit",  # When user submits prompt
    "Stop",  # When stopping execution
    "SubagentStop",  # When a subagent stops
    "TeammateIdle",  # When teammate agent becomes idle (2.1.33+)
    "TaskCompleted",  # When a task finishes execution (2.1.33+)
    "PreCompact",  # Before message compaction
]

**Note**: Python SDK does not support `SessionStart`, `SessionEnd`, or `Notification` hooks due to setup limitations. However, plugins can define `SessionStart` hooks via `hooks.json` using shell commands (e.g., leyline's `detect-git-platform.sh`).

Plugin-Level hooks.json

Plugins can declare hooks via `"hooks": "./hooks/hooks.json"` in plugin.json. The evaluator validates:

• Referenced hooks.json exists and is valid JSON
• Shell commands referenced in hooks exist and are executable
• Hook matchers use valid event types

Hook Callback Signature

async def my_hook(
    input_data: dict[str, Any],  # Hook-specific input
    tool_use_id: str | None,  # Tool ID (for tool hooks)
    context: HookContext,  # Additional context
) -> dict[str, Any]:  # Return decision/messages
    ...

Return Values

return {
    "hookSpecificOutput": {
        "hookEventName": "PreToolUse",  # Match hook type
        "permissionDecision": "deny",  # Optional: block action
        "permissionDecisionReason": "...",  # Reason for denial
        "additionalContext": "...",  # Optional: context added
    }
}

Quality Scoring (100 points)

| Category | Points | Focus | |----------|--------|-------| | Security | 30 | Vulnerabilities, injection, validation | | Performance | 25 | Execution time, memory, I/O | | Compliance | 20 | Structure, documentation, error handling | | Reliability | 15 | Timeouts, idempotency, degradation | | Maintainability | 10 | Code structure, modularity |

Detailed Resources

• **SDK Hook Types**: See `modules/sdk-hook-types.md` for complete Python SDK type definitions, patterns, and examples
• **Evaluation Criteria**: See `modules/evaluation-criteria.md` for detailed scoring rubric and quality gates
• **Security Patterns**: See `modules/sdk-hook-types.md` for vulnerability detection and mitigation
• **Performance Guide**: See `modules/evaluation-criteria.md` for benchmarking and optimization

Basic Evaluation Workflow

# 1. Run detailed evaluation
/hooks-eval --detailed

# 2. Focus on security issues
/hooks-eval --security-only --format sarif

# 3. Benchmark performance
/hooks-eval --performance-baseline

# 4. Check compliance
/hooks-eval --compliance-report

Integration with Other Tools

# Complete plugin evaluation pipeline
/hooks-eval --detailed          # Evaluate all hooks
/analyze-hook hooks/specific.py      # Deep-dive on one hook
/validate-plugin .                   # Validate overall structure

Related Skills

• `abstract:hook-scope-guide` - Decide where to place hooks (plugin/project/global)
• `abstract:hook-authoring` - Write hook rules and patterns
• `abstract:validate-plugin` - Validate complete plugin structure

Troubleshooting

Common Issues

**Hook not firing** Verify hook pattern matches the event. Check hook logs for errors

**Syntax errors** Run `python3 plugins/abstract/scripts/hook_validator.py hooks/hooks.json` (or a Python hook file) before deployment. It parses the file and reports structure and SDK-signature problems.

**Permission denied** Check hook file permissions and ownership

Exit Criteria

• [ ] Every hook in scope receives a composite quality score (0-100) across the five weighted

categories: Security (30), Performance (25), Compliance (20), Reliability (15), Maintainability (10).

• [ ] Any hook scoring below 60 on the Security category is flagged as a blocking issue before

the evaluation report is returned.

• [ ] Shell commands referenced in `hooks.json` are verified to exist and be executable; missing

scripts are listed as FAIL findings.

• [ ] The evaluation report distinguishes between JSON hooks (Claude Code) and Python SDK hooks

and ap