/toolkit

name: toolkit
description: "Toolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md."
user-invocable: true
agent: toolkit-governance-engineer
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash
  - Glob
  - Grep
  - Agent
  - Task
  - Skill
routing:
  force_route: true
  not_for: "application code (use workflow), code review (use review)"
  triggers:
    - "create skill"
    - "create agent"
    - "scaffold skill"
    - "scaffold agent"
    - "new skill"
    - "new agent"
    - "skill template"
    - "agent template"
    - "eval skill"
    - "evaluate agent"
    - "benchmark skill"
    - "benchmark agents"
    - "compare agents"
    - "A/B test agents"
    - "optimize description"
    - "evolve toolkit"
    - "toolkit evolution"
    - "self-improve"
    - "update routing tables"
    - "routing maintenance"
    - "generate claude.md"
    - "create claude.md"
    - "compose skills"
    - "bake-off"
    - "skill quality"
    - "weaker model"
    - "weak model uplift"
    - "improve skill for weaker models"
  category: meta-tooling
  pairs_with:
    - review
    - workflow

Toolkit

Nine modes covering the full toolkit lifecycle: creating and improving skills and agents; evaluating agents; maintaining routing tables; generating CLAUDE.md; composing multi-skill DAGs; and running the evolution loop. Classify the request and follow the matching section.

Mode Selection

| Mode | Signals | Section | |------|---------|---------| | **Skill Creator** | create skill, scaffold skill, new skill, build a skill | Create Skill | | **Agent Creator** | create agent, scaffold agent, new agent | Create Agent | | **Weak-Model Uplift** | weaker model, uplift skill, make a skill work for Opus 4.6, improve guidance from generated output | Uplift for Weaker Models | | **Agent Comparison** | compare agents, A/B test agents, benchmark agents, benchmark skill, bake-off | Compare Agents | | **Agent Evaluation** | evaluate agent quality, audit agent, grade agent, eval skill | Evaluate Agent | | **Skill Composer** | compose skills, DAG orchestration, skill pipeline | Compose Skills | | **Routing Tables** | update routing tables, sync routing, routing drift | Update Routing | | **Toolkit Evolution** | evolve toolkit, self-improve, discover gaps | Evolve Toolkit | | **Generate CLAUDE.md** | generate claude.md, create claude.md, init | Generate CLAUDE.md |

---

Create Skill

Phases: **INTENT -> DRAFT -> TEST -> REGISTER**

1. **Capture intent.** What should the skill do? When should it trigger? What output? Are outputs objectively verifiable (code, data) or subjective (writing, design)? 2. **Duplicate check.** Run `grep -i "<domain>" skills/*/SKILL.md` to check existing coverage. If an umbrella skill covers the domain, add a reference file instead. 3. **Write SKILL.md.** Follow `references/skill-creator/skill-template.md` for frontmatter structure. Apply Dense-Complete Writing standard. Frontmatter must include: name, description, routing (triggers, not_for, category, pairs_with), allowed-tools. 4. **Test.** Try 3 should-trigger, 2 should-not-trigger, and 2 near-miss prompts with the skill loaded. Revise the SKILL.md until routing and output are right. 5. **Register.** Run `python3 scripts/generate-skill-index.py` to update routing.

Load `references/skill-creator.md` for the full workflow. Deep references in `references/skill-creator/` cover progressive disclosure, artifact schemas, complexity tiers, error catalog, enrichment workflow, and more.

Scripts: `scripts/skill-creator/`

---

Create Agent

Phases: **DISCOVER -> DESIGN -> SCAFFOLD -> REGISTER -> VALIDATE**

1. **Discover.** Check for domain overlap: `grep -i "<domain>" agents/*.md`. If an existing agent covers the domain, add a `references/` file instead. 2. **Design.** Decide role type (reviewer/engineer/orchestrator), allowed tools, complexity, triggers (3-6 specific phrases), pairs_with (verify each exists), reference files, description (intent verb + domain + boundary clause), activation cases. 3. **Scaffold.** Write the agent file using `references/agent-creator/agent-frontmatter-template.md`. Follow `docs/PHILOSOPHY.md` for operator context structure. 4. **Register.** Run `python3 scripts/generate-agent-index.py`. 5. **Validate.** Run `python3 scripts/validate-references.py` to check reference file integrity. Test activation with the 3+2+2 prompt set.

Load `references/agent-creator.md` for full phases. Deep references in `references/agent-creator/` cover design patterns, frontmatter template, eval design.

---

Uplift for Weaker Models

Improve a skill, agent, or shared guide until a weaker model produces strong output with it. Load `references/weak-model-uplift.md` and follow its steps:

1. **Pick the target from data.** Query `~/.claude/learning/usage.db` and `learning.db` for heavily used or failing skills. 2. **Build tasks and checks first.** 4–8 tasks plus 1–2 held-out tasks; deterministic checks and a yes/no rubric written before any run. 3. **Run the arms.** No guidance and current guidance, two samples per task minimum, with `python3 scripts/weak_model_run.py`. 4. **Score and look.** Checks, rubric, your own review of every artifact, optional Jev questions on extracted facts. 5. **Turn failures into rules.** Concrete values, before/after examples, runnable checks; delete stale instructions; examples from unrelated products. 6. **Rerun** the guided arm and held-out tasks; stop when gains flatten or after three rounds. 7. **Report and ship** a per-round table with held-out results, cost, and caveats in the PR body.

---

Compare Agents

Controlled benchmarks comparing agent variants on identical tasks.

1. **Select variants.** Identify the agents to compare (2-4 variants). 2. **Design benchmark.** Load `references/agent-comparison/benchmark-tasks.md`. Select 5-10 representative tasks covering the agent's domain. When the result decides a ship, also load `references/blind-ab-tests