/aris-auto-review-loop

name: aris-auto-review-loop
description: Autonomous multi-round research review loop. Repeatedly reviews via Codex MCP, implements fixes, and re-reviews until positive assessment or max rounds reached. Use when user says "auto review loop", "review until it passes", or wants autonomous iterative improvement.
argument-hint: "[topic-or-scope]"
allowed-tools: Bash(*), Read, Grep, Glob, Write, Edit, Agent, Skill, mcp__codex__codex, mcp__codex__codex-reply
license: MIT
metadata:
  author: wanshuiyin/ARIS
  version: "1.0.0"

Auto Review Loop: Autonomous Research Improvement

Autonomously iterate: review → implement fixes → re-review, until the external reviewer gives a positive assessment or MAX_ROUNDS is reached.

Context: $ARGUMENTS

Constants

• MAX_ROUNDS = 4
• POSITIVE_THRESHOLD: score >= 6/10, or verdict contains "accept", "sufficient", "ready for submission"
• REVIEW_DOC: `AUTO_REVIEW.md` in project root (cumulative log)
• REVIEWER_MODEL = `gpt-5.4` — Model used via Codex MCP. Must be an OpenAI model (e.g., `gpt-5.4`, `o3`, `gpt-4o`)
• **HUMAN_CHECKPOINT = false** — When `true`, pause after each round's review (Phase B) and present the score + weaknesses to the user. Wait for user input before proceeding to Phase C. The user can: approve the suggested fixes, provide custom modification instructions, skip specific fixes, or stop the loop early. When `false` (default), the loop runs fully autonomously.
• **COMPACT = false** — When `true`, (1) read `EXPERIMENT_LOG.md` and `findings.md` instead of parsing full logs on session recovery, (2) append key findings to `findings.md` after each round.
• **REVIEWER_DIFFICULTY = medium** — Controls how adversarial the reviewer is. Three levels:
• `medium` (default): Current behavior — MCP-based review, Claude controls what context GPT sees.
• `hard`: Adds **Reviewer Memory** (GPT tracks its own suspicions across rounds) + **Debate Protocol** (Claude can rebut, GPT rules).
• `nightmare`: Everything in `hard` + **GPT reads the repo directly** via `codex exec` (Claude cannot filter what GPT sees) + **Adversarial Verification** (GPT independently checks if code matches claims).

> 💡 Override: `/aris-auto-review-loop "topic" — compact: true, human checkpoint: true, difficulty: hard`

State Persistence (Compact Recovery)

Long-running loops may hit the context window limit, triggering automatic compaction. To survive this, persist state to `REVIEW_STATE.json` after each round:

{
  "round": 2,
  "threadId": "019cd392-...",
  "status": "in_progress",
  "difficulty": "medium",
  "last_score": 5.0,
  "last_verdict": "not ready",
  "pending_experiments": ["screen_name_1"],
  "timestamp": "2026-03-13T21:00:00"
}

**Write this file at the end of every Phase E** (after documenting the round). Overwrite each time — only the latest state matters.

**On completion** (positive assessment or max rounds), set `"status": "completed"` so future invocations don't accidentally resume a finished loop.

Workflow

Initialization

1. **Check for `REVIEW_STATE.json`** in project root:

• If it does not exist: **fresh start** (normal case, identical to behavior before this feature existed)
• If it exists AND `status` is `"completed"`: **fresh start** (previous loop finished normally)
• If it exists AND `status` is `"in_progress"` AND `timestamp` is older than 24 hours: **fresh start** (stale state from a killed/abandoned run — delete the file and start over)
• If it exists AND `status` is `"in_progress"` AND `timestamp` is within 24 hours: **resume**
• Read the state file to recover `round`, `threadId`, `last_score`, `pending_experiments`
• Read `AUTO_REVIEW.md` to restore full context of prior rounds
• If `pending_experiments` is non-empty, check if they have completed (e.g., check screen sessions)
• Resume from the next round (round = saved round + 1)
• Log: "Recovered from context compaction. Resuming at Round N."

2. Read project narrative documents, memory files, and any prior review documents. **When `COMPACT = true` and compact files exist**: read `findings.md` + `EXPERIMENT_LOG.md` instead of full `AUTO_REVIEW.md` and raw logs — saves context window. 3. Read recent experiment results (check output directories, logs) 4. Identify current weaknesses and open TODOs from prior reviews 5. Initialize round counter = 1 (unless recovered from state file) 6. Create/update `AUTO_REVIEW.md` with header and timestamp

Loop (repeat up to MAX_ROUNDS)

Phase A: Review

**Route by REVIEWER_DIFFICULTY:**

Medium (default) — MCP Review

Send comprehensive context to the external reviewer:

mcp__codex__codex:
  config: {"model_reasoning_effort": "xhigh"}
  prompt: |
    [Round N/MAX_ROUNDS of autonomous review loop]

    [Full research context: claims, methods, results, known weaknesses]
    [Changes since last round, if any]

    Please act as a senior ML reviewer (NeurIPS/ICML level).

    1. Score this work 1-10 for a top venue
    2. List remaining critical weaknesses (ranked by severity)
    3. For each weakness, specify the MINIMUM fix (experiment, analysis, or reframing)
    4. State clearly: is this READY for submission? Yes/No/Almost

    Be brutally honest. If the work is ready, say so clearly.

If this is round 2+, use `mcp__codex__codex-reply` with the saved threadId to maintain conversation context.

Hard — MCP Review + Reviewer Memory

Same as medium, but **prepend Reviewer Memory** to the prompt:

mcp__codex__codex:
  config: {"model_reasoning_effort": "xhigh"}
  prompt: |
    [Round N/MAX_ROUNDS of autonomous review loop]

    ## Your Reviewer Memory (persistent across rounds)
    [Paste full contents of REVIEWER_MEMORY.md here]

    IMPORTANT: You have memory from prior rounds. Check whether your
    previous suspicions were genuinely addressed or merely sidestepped.
    The author (Claude) controls what context you see — be skeptical
    of convenient omissions.