/toolkit-evolution

name: toolkit-evolution
description: "Closed-loop toolkit self-improvement: discover gaps, diagnose, propose, critique, build, test, evolve."
user-invocable: true
argument-hint: "<optional: focus area like 'routing' or 'hooks'>"
command: evolve
context: fork
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash
  - Glob
  - Grep
  - Agent
  - Skill
routing:
  triggers:
    - "evolve toolkit"
    - "improve the system"
    - "self-improve"
    - "toolkit evolution"
    - "what should we improve"
    - "find improvement opportunities"
    - "discover skill gaps"
    - "what skills are missing"
    - "systematic improvement"
  pairs_with:
    - multi-persona-critique
    - skill-eval
  complexity: Complex
  category: meta-tooling

Toolkit Evolution

Schedulable (nightly) or manually-invoked 7-phase pipeline for continuous toolkit self-improvement. Discovers gaps, diagnoses problems from evidence, proposes solutions, critiques via multi-persona review, builds winners on isolated branches, A/B tests, and promotes via PR.

Nightly sibling of `auto-dream` (2:07 AM consolidates memories; 3:07 AM this skill diagnoses and builds). They feed each other: dream's consolidated memories and injection payload inform evolution's diagnosis; evolution's results become dream's next input.

Invoke: `/evolve`, `/evolve routing`, `/evolve hooks`, `/evolve --discover`. Cron setup in `references/evolve-preferred-patterns.md` § Scheduling.

Reference Loading Table

| Signal | Load These Files | Why | |---|---|---| | running DISCOVER/DIAGNOSE commands: learning DB queries, git scan, drift checks | `diagnose-scripts.md` | Loads detailed guidance from `diagnose-scripts.md`. | | mining merged-PR history and review comments (Phase 0 Step 2b) | `diagnose-scripts.md` | Read-only `gh pr list`/`gh pr view --comments` commands, § DISCOVER Step 2b | | writing the evolution cycle report | `evolution-report-template.md` | Loads detailed guidance from `evolution-report-template.md`. | | Phase 3 CRITIQUE fallback; failure modes, error handling, cost estimates, cron setup | `evolve-preferred-patterns.md` | Loads detailed guidance from `evolve-preferred-patterns.md`. | | Phase 6 EVOLVE: PR creation, merge, branch cleanup, learning records | `evolve-scripts.md` | Loads detailed guidance from `evolve-scripts.md`. |

Instructions

Phase 0: DISCOVER -- Find what's missing

**Goal**: Identify skills, agents, or capability categories the toolkit should have but doesn't. While later phases improve existing components, this phase finds entirely new capabilities the toolkit is missing.

**Frequency**: Monthly, not every run. The DISCOVER phase only executes if:

• `--discover` flag is passed explicitly, OR
• It has been 30+ days since the last discovery run

Check the last discovery run date using the frequency check command from `references/diagnose-scripts.md` § Discovery Frequency Check.

If neither condition is met, skip directly to Phase 1.

**Step 1: Gather briefing data**

Collect current toolkit state using the briefing data commands from `references/diagnose-scripts.md` § DISCOVER Step 1. Brief all 5 perspective agents with the same baseline.

**Step 2: Dispatch 5 perspective agents in parallel**

See `references/evolve-preferred-patterns.md` § Phase 0 DISCOVER for the full agent table and proposal format. Dispatch all 5 simultaneously.

**Step 2b: Mine merged-PR history**

Read-only `gh` queries over the last 30 merged PRs plus their review-comment threads surface recurring friction, repeated fix patterns, and skill/agent gaps that perspective agents miss because they read current state, not history. Commands and interpretation guide: `references/diagnose-scripts.md` § DISCOVER Step 2b. Tag every surviving proposal `[PR-HISTORY]`.

**Step 3: Deduplicate and filter** -- remove duplicates of existing skills (check `skills/INDEX.json`), remove proposals with no evidence (require at least one concrete data point), group similar proposals and note convergent evidence.

**Step 4: Feed into DIAGNOSE** -- append surviving proposals to the Phase 1 opportunity list with source tagged `[DISCOVER]` (perspective agents) or `[PR-HISTORY]` (PR mining).

**Step 5: Save discovery report** to `evolution-reports/discovery-{YYYY-MM-DD}.md` (run `mkdir -p evolution-reports` first). Include briefing data, all proposals, filtering rationale, forwarded proposals, and date stamp.

**Gate**: Discovery report saved. Proposals forwarded to Phase 1. Proceed to DIAGNOSE.

---

Phase 1: DIAGNOSE -- Find improvement opportunities

**Goal**: Identify 5-10 evidence-backed improvement opportunities from multiple data sources.

**Step 1: Read routing telemetry for weak and failing routes**

Run the queries from `references/diagnose-scripts.md` § DIAGNOSE Step 1.

Look for: routes with many dispatches and a high error rate, skills that consistently underperform, and components carrying zero routes over a long window.

**Step 2: Scan recent git history for patterns**

Run the git history commands from `references/diagnose-scripts.md` § DIAGNOSE Step 2.

**Step 3: Check auto-dream reports for accumulated insights**

Run the dream report check from `references/diagnose-scripts.md` § DIAGNOSE Step 3, then read the most recent dream-analysis file.

**Step 3b: Cross-validate dream insights against current state**

Before treating any dream insight as a proposal signal, verify it still reflects the current repo. Use the cross-validation commands from `references/diagnose-scripts.md` § DIAGNOSE Step 3b.

Mark an insight as STALE if: (a) it names a file that no longer exists, OR (b) it claims recent activity but `git log` shows nothing in the past 7 days.

**Step 4: Check routing-table drift**

Skills present in `skills/INDEX.json` but absent from the routing manifest represent a documentation gap. Run the routing-drift check from `references/diagnose-scripts.md` § DIAGNOSE Step 4.

**Step 4b: Check for orphaned ADR session files**

Run the orphaned ses