/testing-handbook-generator

name: testing-handbook-generator
description: "Generates Claude Code skills from the Trail of Bits Testing Handbook (appsec.guide), analyzing handbook pages and emitting SKILL.md files with the structure each skill type requires. Use when creating or refreshing a skill from handbook content, or when the user names the testing handbook or appsec.guide. Not for answering security testing questions — the generated skills cover those."

Testing Handbook Skill Generator

Generate and maintain Claude Code skills from the Trail of Bits Testing Handbook.

When to Use

**Invoke this skill when:**

• Creating new security testing skills from handbook content
• User mentions "testing handbook", "appsec.guide", or asks about generating skills
• Bulk skill generation or refresh is needed

**Do NOT use for:**

• General security testing questions (use the generated skills)
• Non-handbook skill creation

Handbook Location

The skill needs the Testing Handbook repository. See [discovery.md](discovery.md) for full details.

**Quick reference:** Check `./testing-handbook`, `../testing-handbook`, `~/testing-handbook` → ask user → clone as last resort.

**Repository:** `https://github.com/trailofbits/testing-handbook`

Workflow Overview

Phase 0: Setup              Phase 1: Discovery
┌─────────────────┐        ┌─────────────────┐
│ Locate handbook │   →    │ Analyze handbook│
│ - Find or clone │        │ - Scan sections │
│ - Confirm path  │        │ - Classify types│
└─────────────────┘        └─────────────────┘
         ↓                          ↓
Phase 3: Generation        Phase 2: Planning
┌─────────────────┐        ┌─────────────────┐
│ TWO-PASS GEN    │   ←    │ Generate plan   │
│ Pass 1: Content │        │ - New skills    │
│ Pass 2: X-refs  │        │ - Updates       │
│ - Write to gen/ │        │ - Present user  │
└─────────────────┘        └─────────────────┘
         ↓
Phase 4: Testing           Phase 5: Finalize
┌─────────────────┐        ┌─────────────────┐
│ Validate skills │   →    │ Post-generation │
│ - Run validator │        │ - Update README │
│ - Test activation│       │ - Update X-refs │
│ - Fix issues    │        │ - Self-improve  │
└─────────────────┘        └─────────────────┘

Scope Restrictions

**ONLY modify these locations:**

• `plugins/testing-handbook-skills/skills/[skill-name]/*` - Generated skills (as siblings to testing-handbook-generator)
• `plugins/testing-handbook-skills/skills/testing-handbook-generator/*` - Self-improvement
• Repository root `README.md` - Add generated skills to table

**NEVER modify or analyze:**

• Other plugins (`plugins/property-based-testing/`, `plugins/static-analysis/`, etc.)
• Other skills outside this plugin

Do not scan or pull into context any skills outside of `testing-handbook-skills/`. Generate skills based solely on handbook content and resources referenced from it.

Quick Reference

Section → Skill Type Mapping

| Handbook Section | Skill Type | Template | |------------------|------------|----------| | `/static-analysis/[tool]/` | Tool Skill | tool-skill.md | | `/fuzzing/[lang]/[fuzzer]/` | Fuzzer Skill | fuzzer-skill.md | | `/fuzzing/techniques/` | Technique Skill | technique-skill.md | | `/crypto/[tool]/` | Domain Skill | domain-skill.md | | `/web/[tool]/` | Tool Skill | tool-skill.md |

Skill Candidate Signals

| Signal | Indicates | |--------|-----------| | `_index.md` with `bookCollapseSection: true` | Major tool/topic | | Numbered files (00-, 10-, 20-) | Structured content | | `techniques/` subsection | Methodology content | | `99-resources.md` or `91-resources.md` | Has external links |

Exclusion Signals

| Signal | Action | |--------|--------| | `draft: true` in frontmatter | Skip section | | Empty directory | Skip section | | Template/placeholder file | Skip section | | GUI-only tool (e.g., `web/burp/`) | Skip section (Claude cannot operate GUI tools) |

Decision Tree

**Starting skill generation?**

├─ Need to analyze handbook and build plan?
│  └─ Read: discovery.md
│     (Handbook analysis methodology, plan format)
│
├─ Spawning skill generation agents?
│  └─ Read: agent-prompt.md
│     (Full prompt template, variable reference, validation checklist)
│
├─ Generating a specific skill type?
│  └─ Read appropriate template:
│     ├─ Tool (Semgrep, CodeQL) → templates/tool-skill.md
│     ├─ Fuzzer (libFuzzer, AFL++) → templates/fuzzer-skill.md
│     ├─ Technique (harness, coverage) → templates/technique-skill.md
│     └─ Domain (crypto, web) → templates/domain-skill.md
│
├─ Validating generated skills?
│  └─ Run: scripts/validate-skills.py
│     Then read: testing.md for activation testing
│
├─ Finalizing after generation?
│  └─ See: Post-Generation Tasks below
│     (Update main README, update Skills Cross-Reference, self-improvement)
│
└─ Quick generation from specific section?
   └─ Use Quick Reference above, apply template directly

Two-Pass Generation (Phase 3)

Generation uses a **two-pass approach** to solve forward reference problems (skills referencing other skills that don't exist yet).

Pass 1: Content Generation (Parallel)

Generate all skills in parallel **without** the Related Skills section:

Pass 1 - Generating 5 skills in parallel:
├─ Agent 1: libfuzzer (fuzzer) → skills/libfuzzer/SKILL.md
├─ Agent 2: aflpp (fuzzer) → skills/aflpp/SKILL.md
├─ Agent 3: semgrep (tool) → skills/semgrep/SKILL.md
├─ Agent 4: harness-writing (technique) → skills/harness-writing/SKILL.md
└─ Agent 5: wycheproof (domain) → skills/wycheproof/SKILL.md

Each agent uses: pass=1 (content only, Related Skills left empty)

**Pass 1 agents:**

• Generate all sections EXCEPT Related Skills
• Leave a placeholder: `## Related Skills\n\n<!-- PASS2: populate after all skills exist -->`
• Output report includes `references: DEFERRED`

Pass 2: Cross-Reference Population (Sequential)

After all Pass 1 agents complete, run Pass 2 to populate Related Skills:

Pass 2 - Populating