/result-to-claim

name: result-to-claim
description: Use when experiments complete to judge what claims the results support, what they don't, and what evidence is still missing. Codex MCP evaluates results against intended claims and routes to next action (pivot, supplement, or confirm). Use after experiments finish — before writing the paper or running ablations.
argument-hint: "[experiment-description-or-wandb-run]"
allowed-tools: Bash(*), Read, Grep, Glob, Write, Edit, mcp__codex__codex, mcp__codex__codex-reply, mcp__antigravity__antigravity, mcp__antigravity__antigravity-reply, mcp__grok__grok, mcp__grok__grok-reply, mcp__claude-review__review, mcp__claude-review__review_reply, mcp__claude-review__review_start, mcp__claude-review__review_reply_start, mcp__claude-review__review_status

Result-to-Claim Gate

> **Before each reviewer slot**, follow [Selectable Reviewers](../shared-references/reviewer-backends.md): > resolve invocation → `.aris/reviewer.json` → Claude Code default (Codex MCP) > with `reviewer_route.py` through the canonical helper resolver, preserving this > slot's tier, fresh contexts, source/image inputs and deterministic gates. > Codex examples/pins below apply only to `backend=codex`; other routes use their > own first-call/reply contract. Save the route and actual returned identity; > unavailable selections are `REVIEW_UNAVAILABLE`. Classify acceptance from actual > families/completion/scope. Forward invocation reviewer/model/effort overrides > to reviewer-bearing children. Generation workers are unchanged.

> 🔒 **Do not wrap this skill in `/loop`, `/schedule`, or `CronCreate`.** It is > verdict-bearing — it judges whether results support a claim. Re-running that > verdict on a wall-clock timer adds no new signal (the verdict changes only > when the *results* change, not when the clock ticks). What you actually want > to schedule is the *external wait that precedes it* — experiments done → then > run this gate **once**. See > [`shared-references/external-cadence.md`](../shared-references/external-cadence.md).

Experiments produce numbers; this gate decides what those numbers *mean*. Collect results from available sources, get a Codex judgment, then auto-route based on the verdict.

Context: $ARGUMENTS

When to Use

• After a set of experiments completes (main results, not just sanity checks)
• Before committing to claims in a paper or review response
• When results are ambiguous and you need an objective second opinion

Workflow

Step 1: Collect Results

Gather experiment data from whatever sources are available in the project:

1. **W&B** (preferred): `wandb.Api().run("<entity>/<project>/<run_id>").history()` — metrics, training curves, comparisons 2. **EXPERIMENT_LOG.md**: full results table with baselines and verdicts 3. **EXPERIMENT_TRACKER.md**: check which experiments are DONE vs still running 4. **Log files**: `ssh server "tail -100 /path/to/training.log"` if no other source 5. **`idea-stage/docs/research_contract.md`** (legacy fallback: `docs/research_contract.md`): intended claims and experiment design

Assemble the key information:

• What experiments were run (method, dataset, config)
• Main metrics and baseline comparisons (deltas)
• The intended claim these experiments were designed to test
• Any known confounds or caveats

Step 1.5: Deterministic evidence pre-check (before spending a Codex call)

For every claim that cites a specific number + a source file, verify the evidence *exists* mechanically — no model call — to catch **hallucinated evidence** before the jury runs (see [`shared-references/evidence-precheck.md`](../shared-references/evidence-precheck.md)).

**1. Build the claims list.** From the cited numbers and their result files, write `[{"id", "value", "source"}, ...]` to `.aris/claims.json` (`source` is the result file/glob relative to the project root; `value` is the cited number or string).

**2. Run the pre-check — this is a real step, not a suggestion.** Execute the block below (resolver per integration-contract §2, **Policy B**: warn-and-skip if the helper is unresolved — never block the audit):

# Policy B = warn-and-skip: nothing here may abort the audit. cd is non-fatal, the
# helper run is explicitly non-blocking, no pipefail-fragile pipe.
cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" 2>/dev/null || true
if [ -z "${ARIS_REPO:-}" ] && [ -f .aris/installed-skills.txt ]; then
    ARIS_REPO=$(awk -F'\t' '$1=="repo_root"{print $2; exit}' .aris/installed-skills.txt 2>/dev/null) || true
fi
if [ -z "${ARIS_REPO:-}" ] && [ -f "$HOME/.aris/repo" ]; then
    ARIS_REPO=$(cat "$HOME/.aris/repo" 2>/dev/null) || true
fi
EVIDENCE_CHECK=".aris/tools/evidence_check.py"
[ -f "$EVIDENCE_CHECK" ] || EVIDENCE_CHECK="tools/evidence_check.py"
[ -f "$EVIDENCE_CHECK" ] || { [ -n "${ARIS_REPO:-}" ] && EVIDENCE_CHECK="$ARIS_REPO/tools/evidence_check.py"; }
[ -f "$EVIDENCE_CHECK" ] || EVIDENCE_CHECK=""

mkdir -p .aris
if [ -n "$EVIDENCE_CHECK" ]; then
    # NB: evidence_check exits 1 when it FINDS hallucinated evidence (value_not_found /
    # path_missing) — that is the useful signal, NOT a failure. So judge success by
    # whether valid JSON was produced, never by exit code. `|| true` keeps set -e calm.
    python3 "$EVIDENCE_CHECK" . --batch .aris/claims.json > .aris/evidence_precheck.json 2>.aris/evidence_precheck.err || true
    if [ -s .aris/evidence_precheck.json ] && python3 -c "import json,sys;json.load(open('.aris/evidence_precheck.json'))" 2>/dev/null; then
        cat .aris/evidence_precheck.json
    else
        echo "WARN: evidence_check produced no valid output (see .aris/evidence_precheck.err);" >&2
        echo "      pre-check skipped (Policy B); the Codex jury still runs." >&2
    fi
else
    echo "WARN: evidence_check.py not resolved at .aris/tools/, tools/, \$ARIS_REPO/tools/, or via ~/.aris/repo." >&2
    echo "      Pre-check skipped (Policy B); the Codex jury still runs. Fix: rerun" >&2
    echo "      bash