/self-evolution

name: self-evolution
version: 4.0.0
description: >
  Iteratively evolve any measurable artifact (prompt, skill, code, idea, configuration,
  document, benchmarked experiment) through autonomous mutation-evaluate-gate loops.
  Supports both GT case suites and autoresearch-style scalar metric loops where a fixed
  command prints one score. Uses 8-phase iteration, 3-layer evaluation, deterministic
  keep/discard gates, trace-driven diagnosis, and layered mutation. Trigger on "evolve
  this", "optimize iteratively", "self-improve", "train this prompt", "iterate on this
  code", "make this better through iteration", "run evolution loop", "self-evolution",
  "evolve this skill", "train this skill", "optimize skill quality", "autonomous
  research", "autoresearch", or whenever the user has a measurable artifact that needs
  data-driven improvement with automatic keep/revert decisions.

Evolve

You are the evolution controller. The user provides an artifact and a definition of "good" — either Ground Truth (GT) cases or a scalar metric. You drive the loop: mutate, evaluate, gate, keep or revert, repeat.

**Core principle:** Any artifact that can be evaluated can be trained. You need three things: 1. **Artifact** — the thing being improved (a prompt, a skill, code, an idea document, a config, an experiment) 2. **Oracle** — GT cases or a scalar metric that defines what "better" means 3. **Execution method** — how to produce output or a score from the artifact for evaluation

Operating Modes

Choose the lightest loop that still has a real oracle:

1. **GT Suite Mode** — Use when quality is defined by multiple test cases or assertions. This is the default for prompts, skills, documents, configs, and broad behavior. 2. **Scoreboard Mode** — Use when a fixed command or harness prints one primary metric (accuracy, loss, val_bpb, latency, score). This is the autoresearch pattern: mutate one bounded surface, run the benchmark, keep only metric-improving changes, repeat. 3. **Hybrid Mode** — Use when a scalar metric is primary but regressions matter. Gate on the primary metric, plus a small regression suite for safety or correctness.

Do not force GT case generation when the user already has a fixed executable metric. A scalar metric with direction, command, editable scope, and hard constraints is enough to start.

Prerequisites

Before starting, verify: 1. The artifact exists and is identifiable (a file, a directory, or a clearly scoped text block) 2. An oracle exists:

• GT test cases exist or will be generated (see `references/ground-truth.md`), OR
• a scalar metric command exists with metric name, direction (`minimize` or `maximize`), and parse rule

3. The editable scope and forbidden scope are explicit 4. The artifact's parent directory is under git (or you will init it) 5. An execution method is clear for the artifact type (see `references/artifact-guide.md`)

If neither GT nor scalar metric exists, help the user create one. Analyze the artifact, propose 10-15 test cases with assertions or one benchmark command with metric extraction, and ask for review.

**For skills specifically:** If `skill-creator` is installed, use `quick_validate.py` for L1 validation and skill-creator's grader for L2 evaluation. See `references/artifact-guide.md` for skill-specific execution methods.

---

Phase 0 — Setup (One-Time)

0.1 Classify the Artifact

Determine the artifact type and run mode. This drives execution method and mutation layer definitions.

| Type | Artifact Is | Execution Method | Default Mode | Example | |------|------------|------------------|--------------|---------| | `prompt` | A text prompt/instruction | Send to LLM with test input, capture output | GT Suite | System prompt, few-shot template | | `skill` | SKILL.md + references/scripts | Run claude with skill loaded | GT Suite | Claude Code skill | | `code` | Source code files | Run/test via shell command | GT Suite or Scoreboard | Python function, JS module | | `experiment` | Bounded code/config optimized by one benchmark | Run fixed command, parse scalar metric | Scoreboard | autoresearch `train.py` | | `idea` | A document/proposal | LLM evaluates against criteria | GT Suite | Business plan, design doc | | `config` | Configuration file | Apply config, run system, check behavior | GT Suite or Scoreboard | YAML config, .env settings | | `custom` | User-defined | User provides execution command | GT Suite, Scoreboard, or Hybrid | Anything else |

If the artifact or oracle is ambiguous, ask the user. If only the run mode is ambiguous, prefer GT Suite for semantic quality and Scoreboard for numeric optimization.

0.2 Define the Oracle and Scope Contract

Record the contract before the first mutation:

• **Editable scope:** exact file(s), directories, or text blocks the loop may change
• **Forbidden scope:** files, harnesses, generated outputs, dependencies, external services, or contracts that must not change
• **Oracle command:** the command, API call, evaluation procedure, or LLM grading method that produces the result
• **Metric:** pass_rate for GT Suite, or scalar metric name + direction for Scoreboard
• **Budget:** timeout, token/time ceiling, memory ceiling, or "none" if not relevant
• **Keep rule:** what counts as progress
• **Discard rule:** crashes, regressions, worse metrics, safety failures, or constraint violations

The oracle is the source of truth. Do not edit the oracle to make a mutation pass.

0.3 Workspace Initialization

Create workspace as a sibling to the artifact:

<artifact-name>-evolution/
├── evolve_plan.md          # Strategy document
├── results.tsv             # Per-iteration summary (append-only)
├── experiments.jsonl        # Structured experiment log (append-only)
├── gt/                     # GT Suite / Hybrid only
│   ├── dev.json
│   ├── holdout.json
│   └── regression.json
├── traces/                 # Per-iteration execution traces
└── iterations/             # Per-iteration s