/auto-review-loop

name: auto-review-loop
description: Adversarial iterative review loop with generator-evaluator separation. Up to 4 rounds of independent review, improvement, and re-evaluation. Persists state to output/REVIEW_STATE.json for recovery. Stop: score ≥ 7.5/10 on all dimensions, or 4 rounds. Writes full history to output/AUTO_REVIEW_REPORT.md.
argument-hint: [topic-or-scope]
tools: all
flags:
  HUMAN_CHECKPOINT: true    # If true, pause after each round for user approval of fixes
  COMPACT_MODE: false       # If true, use output/EXPERIMENT_RESULT.md instead of full experiment logs
  EXTERNAL_REVIEW: false    # If true, use MCP claude-review or gemini-review server

Skill: auto-review-loop

You run adversarial review cycles to iteratively improve research work. The architecture enforces **generator-evaluator separation**: the entity that wrote a section does NOT score it. Each round: independent review → parse → implement fixes → re-evaluate → decide.

---

Context: $ARGUMENTS

Core Principle: Generator-Evaluator Separation

**Never let the same agent that wrote a section score it.**

Constants

• MAX_ROUNDS = 4
• POSITIVE_THRESHOLD: score >= 6/10, or verdict contains "accept", "sufficient", "ready for submission"
• REVIEW_DOC: `output/AUTO_REVIEW_REPORT.md` in project root (cumulative log)
• REVIEWER_MODEL = `gpt-5.4` — Preferred external reviewer via Codex MCP (OpenAI models: `gpt-5.4`, `o3`, `gpt-4o`, etc.). If Codex MCP / `codex exec` is unavailable or misconfigured, fall back to `REVIEWER_FALLBACK` (see Phase A.0).
• REVIEWER_FALLBACK = `claude-opus-4-6` — The most powerful Claude model, invoked via the `Agent` tool in a **separate context** as the adversarial reviewer subagent. Used automatically when the external LLM cannot be reached.
• RESEARCH_DOMAIN = `auto` — One of `ml`, `giscience`, `remote-sensing`, `spatial-data-science`, or `auto` (infer from the work under review). Drives the reviewer persona, rubric emphasis, and domain-specific must-checks (see Phase A.5).
• **HUMAN_CHECKPOINT = false** — When `true`, pause after each round's review (Phase B) and present the score + weaknesses to the user. Wait for user input before proceeding to Phase C. The user can: approve the suggested fixes, provide custom modification instructions, skip specific fixes, or stop the loop early. When `false` (default), the loop runs fully autonomously.
• **COMPACT = false** — When `true`, (1) read `output/EXPERIMENT_LOG.md` and `output/PROJ_NOTES.md` instead of parsing full logs on session recovery, (2) append key findings to `output/PROJ_NOTES.md` after each round.
• **REVIEWER_DIFFICULTY = medium** — Controls how adversarial the reviewer is. Three levels:
• `medium` (default): Current behavior — MCP-based review, Claude controls what context GPT sees.
• `hard`: Adds **Reviewer Memory** (GPT tracks its own suspicions across rounds) + **Debate Protocol** (Claude can rebut, GPT rules).
• `nightmare`: Everything in `hard` + **GPT reads the repo directly** via `codex exec` (Claude cannot filter what GPT sees) + **Adversarial Verification** (GPT independently checks if code matches claims).

> 💡 Override: `/auto-review-loop "topic" — compact: true, human checkpoint: true, difficulty: hard`

State Persistence (Compact Recovery)

Long-running loops may hit the context window limit, triggering automatic compaction, or crash mid-round. To survive both, persist state to `output/REVIEW_STATE.json` **at every phase boundary** (not only end-of-round):

{
  "round": 2,
  "phase": "C_implementing_fixes",
  "threadId": "019cd392-...",
  "status": "in_progress",
  "difficulty": "medium",
  "persona": "remote-sensing",
  "scores_per_round": [
    {"round": 1, "score": 5.0, "verdict": "not ready"}
  ],
  "last_score": 5.0,
  "last_verdict": "not ready",
  "open_weaknesses": [
    {"id": "W1", "title": "no spatial CV", "first_seen_round": 1, "status": "open"}
  ],
  "pending_experiments": ["screen_name_1"],
  "round_started_at": "2026-03-13T21:00:00",
  "loop_started_at": "2026-03-13T18:00:00",
  "timestamp": "2026-03-13T21:14:03"
}

**Write rules:**

• Overwrite `output/REVIEW_STATE.json` at each phase transition (A → B → B.5 → B.6 → C → D → E). `phase` values: `A_selecting_backend`, `A5_selecting_persona`, `A_reviewing`, `B_parsing`, `B5_memory_update`, `B6_debate`, `C_implementing_fixes`, `D_waiting_results`, `E_documenting`.
• `scores_per_round` is an append-only array — never overwrite prior round scores.
• `open_weaknesses` entries carry `first_seen_round` so the circuit breaker in Phase B.7 can detect repeats.

**On completion** (positive assessment or max rounds or blocked), set `"status"` to `"completed"`, `"max_rounds_reached"`, or `"blocked"` so future invocations don't resume a finished loop.

Budget Caps (safety valves)

Hard caps that terminate the loop with `status: "blocked"` and a clear reason written to `output/AUTO_REVIEW_REPORT.md`:

• **MAX_ROUNDS = 4** (as before)
• **MAX_WALL_CLOCK_HOURS = 6** — total elapsed from `loop_started_at` to current time. Prevents runaway overnight loops. Override via argument (`max-hours: N`).
• **MAX_ROUND_WALL_CLOCK_HOURS = 2** — any single round exceeding this triggers a soft warning in the round log and a user notification; the round is not killed automatically (experiments may legitimately be long), but the cap is enforced cumulatively by `MAX_WALL_CLOCK_HOURS`.
• **MAX_STALLED_ROUNDS = 2** — see Phase B.7.

Log `round_started_at` at the top of Phase A and compute elapsed at Phase E.

Workflow

Initialization

1. **Check for `output/REVIEW_STATE.json`** in project root:

• If it does not exist: **fresh start** (normal case, identical to behavior before this feature existed)
• If it exists AND `status` is `"completed"`: **fresh start** (previous loop finished normally)
• If it exists AND `status` is `"in_progress"` AND `timestamp` is older than 24 hours: **fresh start** (stale state from a killed/abandoned run —