/retro

name: retro
description: 'Post-run retrospective: reads .experiments/ JSONL, computes Wilcoxon significance, detects dead iterations, flags suspicious jumps, generates next-hypothesis queue for --hypothesis flag.'
argument-hint: '[<run-id>] [--compare <run-id-2>] [--threshold <delta>] [--alpha <significance>]'
effort: medium
allowed-tools: Read, Write, Bash, Glob, Agent, TaskCreate, TaskUpdate, AskUserQuestion
disable-model-invocation: true

<objective>

Post-run retrospective analysis. After `/research:run` completes, reads `.experiments/state/<run-id>/experiments.jsonl`, computes statistical significance, detects dead iterations, flags suspicious metric jumps, generates learning summary with next-hypothesis queue.

NOT for: running experiments (use `/research:run`); designing experiments (use `/research:plan`); validating methodology (use `/research:judge`); verifying paper implementation (use `/research:verify`); comparing runs from different programs/goals — `--compare` valid only for same-program, same-metric runs. Read-only — never modifies code, commits, or experiment state.

</objective>

<workflow>

Agent Resolution

**Agent resolution**: load and follow the protocol below. Contains: foundry check + fallback table. `research:scientist` in same plugin — no fallback needed if research plugin installed.

PLUGIN_ROOT=$(python "${CLAUDE_PLUGIN_ROOT}/bin/resolve_plugin_root.py" --plugin-name research) || exit $?; export PLUGIN_ROOT
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
# loads: unsupported-flag-protocol.md
python "$PLUGIN_ROOT/bin/resolve_shared.py" --persist research-shared --emit agent-resolution.md --emit unsupported-flag-protocol.md || exit 1  # timeout: 5000 — cold resolve; every later site reads the research-shared sentinel instead of re-running python

Retro Mode (Steps T1–T7)

**Defaults**: `--threshold 0.001`, `--alpha 0.05`.

**Unsupported flag check**: follow the protocol loaded in the Agent Resolution block. Supported flags for this skill: `--compare`, `--threshold`, `--alpha`.

**Task tracking**: create tasks for T1–T7 at start, all in the same response as the first real tool call. Every later `TaskUpdate` rides with the next real tool call — never a bookkeeping-only turn; the one standalone call is a `completed` right before a long report.

Step T1: Locate and load run data

**Input resolution** (priority order):

1. Explicit `<run-id>` arg → read `.experiments/state/<run-id>/` 2. No arg → scan `.experiments/state/`, pick latest dir where `state.json` has `status: completed` or `status: goal-achieved` 3. None found → stop with error:

   No completed run found. Run /research:run first, or provide: /research:retro <run-id>

**Newer-in-progress check** (only when path 2 used — no explicit run-id given): after selecting completed run, scan `.experiments/state/` for any dir with `status: running` and mtime newer than selected dir. If found, surface warning but don't stop — user may intentionally retro prior completed run:

⚠ Newer in-progress run found: <newer-run-id> (status: running, started <ISO timestamp>). Retro will analyse <selected-run-id> instead. Use /research:retro <run-id> to override.

**Load files** from `.experiments/state/<run-id>/`:

• `state.json`: extract `goal`, `best_metric`, `config` (incl. `metric.direction`), `iteration` count, `best_commit`. Compute `baseline_metric` from iteration 0 in `experiments.jsonl`.
• `experiments.jsonl`: full iteration history — validate each line parses as JSON. If last line truncated, the T1 setup block below warns and drops it from the **sanitized copy `$RUN_DIR/experiments-clean.jsonl`** — produced mechanically once `$RUN_DIR` exists (never via the Write tool, never under `.experiments/state/`), path persisted to `retro-jsonl-path-${CSID}`. All downstream steps (T2 retro_analyze.py, T3 dead-iter scan, T5 scientist) must read sanitized copy — never raw file — so every step sees same iteration set. If JSONL untruncated, sanitized copy byte-identical to raw file.
• `diary.md`: if present, read for qualitative context in T5.

If `--compare <run-id-2>` present: load second run identically from `.experiments/state/<run-id-2>/` — including its own sanitized copy `$RUN_DIR/experiments-clean-2.jsonl` (same truncation rule, produced by the T1 setup block, path in `retro-jsonl2-path-${CSID}`); the compare step reads that copy, never the raw file. If not found, stop: `"Compare target not found: .experiments/state/<run-id-2>/. Check run ID and retry."`

**T1 setup** — one block: assign `RUN_ID_ARG` from `$ARGUMENTS` (first positional non-flag token, empty if absent), resolve `$RUN_ID` (input resolution above), pre-compute and persist `$RUN_DIR` for T3:

PLUGIN_ROOT=$(python "${CLAUDE_PLUGIN_ROOT}/bin/resolve_plugin_root.py" --plugin-name research) || exit $?; export PLUGIN_ROOT
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
# retro has no boolean flag; --flags must be non-empty, so a placeholder name. --keep-state: no eval; --inlined: retro keeps no compaction contract, so it clears none
python "$PLUGIN_ROOT/bin/parse-skill-flags.py" --flags retro --value-flags compare,threshold,alpha --keep-state retro --inlined -- "$ARGUMENTS" >/dev/null  # timeout: 5000
IFS= read -r CLEAN_ARGS < "${TMPDIR:-/tmp}/retro-clean-args-${CSID}" 2>/dev/null || CLEAN_ARGS=""
IFS= read -r VALUE_ALPHA < "${TMPDIR:-/tmp}/retro-alpha-${CSID}" 2>/dev/null || VALUE_ALPHA=""
IFS= read -r VALUE_COMPARE < "${TMPDIR:-/tmp}/retro-compare-${CSID}" 2>/dev/null || VALUE_COMPARE=""
RUN_ID_ARG=$(echo "$CLEAN_ARGS" | awk '{for (i=1; i<=NF; i++) if ($i !~ /^--/) { print $i; exit }}')
RUN_ID_ARG="${RUN_ID_ARG:-}"
echo "$RUN_ID_ARG" > "${TMPDIR:-/tmp}/retro-run-id-${CSID}"  # persist for T3 (vars lost between Bash calls)
echo "${VALUE_ALPHA:-0.05}" > "${TMPDIR:-/tmp}/retro-alpha-${CSID}"
RUN_ID="${RUN_ID_ARG:-$(python "$PLUGIN_ROOT/bin/find_run_id.py" .experiments/state 2>/dev/null)}"  # loads: find_r