/retro

name: retro
description: "Post-run retrospective: reads .experiments/ JSONL, computes Wilcoxon significance, detects dead iterations, flags suspicious jumps, generates next-hypothesis queue for --hypothesis flag."
argument-hint: "[<run-id>] [--compare <run-id-2>] [--threshold <delta>] [--alpha <significance>]"
effort: medium
allowed-tools: Read, Write, Bash, Grep, Glob, Agent, TaskCreate, TaskUpdate, AskUserQuestion
disable-model-invocation: true

<objective>

Post-run retrospective analysis. After `/research:run` completes, reads `.experiments/state/<run-id>/experiments.jsonl`, computes statistical significance, detects dead iterations, flags suspicious metric jumps, generates learning summary with next-hypothesis queue.

NOT for: running experiments (use `/research:run`); designing experiments (use `/research:plan`); validating methodology (use `/research:judge`); verifying paper implementation (use `/research:verify`); comparing runs from different programs/goals — `--compare` valid only for same-program, same-metric runs. Read-only — never modifies code, commits, or experiment state.

</objective>

<workflow>

Agent Resolution

**Agent resolution**: load and follow the protocol below. Contains: foundry check + fallback table. `research:scientist` in same plugin — no fallback needed if research plugin installed.

_RESEARCH_SHARED=$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/resolve_shared.py" 2>/dev/null)  # timeout: 5000
[ -z "$_RESEARCH_SHARED" ] && { echo "! Plugin path resolution failed — ensure research plugin installed and CLAUDE_PLUGIN_ROOT set, or invoke /research:retro from project root."; exit 1; }
cat "$_RESEARCH_SHARED/agent-resolution.md"

Retro Mode (Steps T1–T7)

Triggered by `retro`, `retro <run-id>`, or `retro <run-id> --compare <run-id-2>`.

**Defaults**: `--threshold 0.001`, `--alpha 0.05`.

**Unsupported flag check**: load and follow the protocol below. Supported flags for this skill: `--compare`, `--threshold`, `--alpha`.

# loads: unsupported-flag-protocol.md
_RESEARCH_SHARED=$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/resolve_shared.py" 2>/dev/null)  # timeout: 5000
cat "$_RESEARCH_SHARED/unsupported-flag-protocol.md"

**Task tracking**: create tasks for T1–T7 at start — before any tool calls.

Step T1: Locate and load run data

**Input resolution** (priority order):

1. Explicit `<run-id>` arg → read `.experiments/state/<run-id>/` 2. No arg → scan `.experiments/state/`, pick latest dir where `state.json` has `status: completed` or `status: goal-achieved` 3. None found → stop with error:

   No completed run found. Run /research:run first, or provide: /research:retro <run-id>

**Newer-in-progress check** (only when path 2 used — no explicit run-id given): after selecting completed run, scan `.experiments/state/` for any dir with `status: running` and mtime newer than selected dir. If found, surface warning but don't stop — user may intentionally retro prior completed run:

⚠ Newer in-progress run found: <newer-run-id> (status: running, started <ISO timestamp>). Retro will analyse <selected-run-id> instead. Use /research:retro <run-id> to override.

**Load files** from `.experiments/state/<run-id>/`:

• `state.json`: extract `goal`, `best_metric`, `config` (incl. `metric.direction`), `iteration` count, `best_commit`. Compute `baseline_metric` from iteration 0 in `experiments.jsonl`.
• `experiments.jsonl`: full iteration history — validate each line parses as JSON. If last line truncated, warn and **rewrite sanitized copy to `$RUN_DIR/experiments-clean.jsonl`** (skip truncated last line). All downstream steps (T2 retro_analyze.py, T3 dead-iter scan, T5 scientist) must read sanitized copy — never raw file — so every step sees same iteration set. Persist sanitized path: `echo "$RUN_DIR/experiments-clean.jsonl" > "${TMPDIR:-/tmp}/retro-jsonl-path-${CSID}"` (consumers re-hydrate from this file). If JSONL untruncated, sanitized copy byte-identical to raw file.
• `diary.md`: if present, read for qualitative context in T5.

If `--compare <run-id-2>` present: load second run identically from `.experiments/state/<run-id-2>/`. If not found, stop: `"Compare target not found: .experiments/state/<run-id-2>/. Check run ID and retry."`

**Assign `RUN_ID_ARG`** from `$ARGUMENTS` — first positional non-flag token, empty if absent (ADV-H17):

export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
_REMAINDER=$(echo "$ARGUMENTS" | sed -E 's/--compare[= ]+[^ ]+//g; s/--threshold[= ]+[^ ]+//g; s/--alpha[= ]+[^ ]+//g')
RUN_ID_ARG=$(echo "$_REMAINDER" | awk '{for (i=1; i<=NF; i++) if ($i !~ /^--/) { print $i; exit }}')
RUN_ID_ARG="${RUN_ID_ARG:-}"
echo "$RUN_ID_ARG" > "${TMPDIR:-/tmp}/retro-run-id-${CSID}"  # persist for T3 (vars lost between Bash calls)

**Pre-compute run directory** — also fix `$RUN_ID` (resolved from input resolution above), persist `$RUN_DIR` for T3 (ADV-H18 + ADV-L16):

export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
RUN_ID="${RUN_ID_ARG:-$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/find_run_id.py" .experiments/state 2>/dev/null)}"  # loads: find_run_id.py
# T-G2: find_run_id.py errors suppressed by 2>/dev/null; surface empty case to avoid double-slash path
[ -z "$RUN_ID" ] && { echo "! Failed to resolve run ID — no completed run found or bin/find_run_id.py unavailable; check research plugin install."; exit 1; }
BRANCH=$(git branch --show-current 2>/dev/null | tr '/' '-' || echo 'main')  # timeout: 3000
echo "$RUN_ID" > "${TMPDIR:-/tmp}/retro-run-id-resolved-${CSID}"

export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
RUN_DIR=$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/make_run_dir.py" "retro" ".experiments" 2>/dev/null)  # timeout: 5000
mkdir -p "$RUN_DIR/scripts"  # timeout: 3000
echo "$RUN_DIR" > "${TMPDIR:-/tmp}/retro-run-dir-${CSID}"  # T3 + fallback path reload from temp file

Step T2: Statistical significance analys