/judge

name: judge
description: Research-supervisor review of program.md — validates experimental methodology (hypothesis clarity, measurement validity, control adequacy, scope, strategy fit), emits APPROVED / NEEDS-REVISION / BLOCKED verdict before expensive run loop.
argument-hint: '[<program.md>] [--skip-validation] [--keep "<items>"]'
effort: medium
allowed-tools: Read, Write, Bash, Glob, Agent, Skill, TaskCreate, TaskUpdate, AskUserQuestion
disable-model-invocation: true

<objective>

Research-supervisor review of `program.md` — validates experimental methodology, emits APPROVED / NEEDS-REVISION / BLOCKED verdict before expensive run loop. Read-only apart from its own run dir and report; J4 executes the program's own metric/guard commands, so working-tree drift is checked and reported.

NOT for: running experiments (use `/research:run`); designing hypotheses (use `research:scientist` agent); config quality (`/foundry:audit` (requires `foundry` plugin)). </objective>

<compaction>

• Key boundary: end of J3 — methodology and scientific review agents complete, output files written; before J4 validation and J6 verdict.
• Preserve: RUN_DIR (TMPDIR key), PROGRAM_PATH (TMPDIR key), methodology.md path, scientific-review.md path, SKIP_VALIDATION flag, BRANCH.
• Clear at J1 start (stale prior run) and at end of J6 after terminal summary printed.

</compaction>

<workflow>

<!-- Agent resolution: see _RESEARCH_SHARED/agent-resolution.md -->

Agent Resolution

**Agent resolution**: load and follow the protocol below. Agent this skill dispatches: `research:scientist` (same plugin — always present while this skill runs, so no fallback row).

# loads: compaction-contract.md
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/resolve_shared.py" --persist research-shared --emit agent-resolution.md || exit 1  # timeout: 5000 — cold resolve; every later site reads the research-shared sentinel instead of re-running python

Judge Mode (Steps J1–J6)

Triggered by `judge` or `judge <file.md>`.

**Task tracking**: create tasks for J1, J2, J3, J4, J5a, J5b, J6 at start, all in the same response as the first real tool call. Every later `TaskUpdate` rides with the next real tool call — never a bookkeeping-only turn; the one standalone call is a `completed` right before a long report. (J5a = Codex adversarial review; J5b = resolve rating source.)

Step J1: Locate and parse program.md

**Flag parsing** (first):

export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
[ -f "${TMPDIR:-/tmp}/sweep-out-path-${CSID}" ] && _INL=--inlined || _INL=""  # sweep S1 sentinel present = inlined in sweep: sweep's live contract must survive
python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/parse-skill-flags.py" --flags skip-validation --keep-state judge $_INL "$ARGUMENTS" >/dev/null  # timeout: 5000 — no eval; judge-{skip-validation,clean-args,keep-items}-${CSID} (J3 note block reads skip-validation in a fresh shell), clears a stale contract (not when --inlined)
IFS= read -r SKIP_VALIDATION < "${TMPDIR:-/tmp}/judge-skip-validation-${CSID}" 2>/dev/null || SKIP_VALIDATION=false
IFS= read -r ARGUMENTS < "${TMPDIR:-/tmp}/judge-clean-args-${CSID}" 2>/dev/null || ARGUMENTS=""
echo "SKIP_VALIDATION=${SKIP_VALIDATION:-false}"

**Unsupported flag check**: load and follow the protocol below. Supported flags for this skill: `--skip-validation`, `--keep`.

# loads: unsupported-flag-protocol.md
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
IFS= read -r _RESEARCH_SHARED < "${TMPDIR:-/tmp}/research-shared-${CSID}" 2>/dev/null || _RESEARCH_SHARED=""  # warm read (Check 41)
cat "$_RESEARCH_SHARED/unsupported-flag-protocol.md"

**Input resolution** (priority order):

1. Explicit argument: `/research:judge path/to/plan.md` 2. Auto-detect: `program.md` at project root 3. Latest state: scan `.experiments/state/*/state.json` for most recent with `status: running` and non-null `program_file` field 4. If nothing found: stop with error:

   No program.md found. Run /research:plan <goal> first, or provide a path: /research:judge <path.md>

**Schema gate** — before parsing, count matching `## <Section>` headings against the known schema set (`Goal`, `Metric`, `Guard`, `Config`, `Notes`). Zero matches = located file doesn't parse as program.md (e.g. input resolution located an unrelated document) — stop immediately with the same "nothing found" error shown in step 4 above; do NOT proceed to J2's C1–C13 checklist. Distinct from step 4 (file not locatable at all): this fires when a file IS located but content doesn't match expected schema.

> 1-2 matching headings is a partial match, not a gap here — deferred to J2's C1-C4 criticals (missing Goal / Metric.command / Metric.direction / Guard.command → BLOCKED). A non-program file is reported as a flawed program, by design, not treated as unparsable input.

**Persist program path** — once the schema gate passes, substitute the resolved program.md path into the block below. It aborts when the path is left unsubstituted or is not a file; J3 and J5a re-hydrate it (fresh shell per call):

export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
PROGRAM_FILE="<resolved program.md path>"
case "$PROGRAM_FILE" in *'<'*'>'*|"") echo "! BLOCKED — program path not substituted in J1"; exit 1;; esac
[ -f "$PROGRAM_FILE" ] || { echo "! BLOCKED — program file not found: $PROGRAM_FILE"; exit 1; }
echo "$PROGRAM_FILE" > "${TMPDIR:-/tmp}/judge-program-file-${CSID}"  # J3 pre-spawn block reads it back

**Parsing** — find `## <Section>` headings in program.md, extract first fenced code block per section, parse as `key: value` lines, warn on unrecognized keys. `--skip-validation` and `colab_hw` judge-specific, extracted independently.

**No placeholder substitution** — `metric_cmd` and `guard_cmd` are validated exactly as written: run executes them literally (R1 resolves no `{field_name}` tokens), so substituting here