/result-to-claim

name: result-to-claim
description: Use when experiments complete to judge what claims the results support, what they don't, and what evidence is still missing. Codex MCP evaluates results against intended claims and routes to next action (pivot, supplement, or confirm). Use after experiments finish — before writing the paper or running ablations.
argument-hint: "[experiment-description-or-wandb-run]"
allowed-tools: Bash(*), Read, Grep, Glob, Write, Edit, mcp__codex__codex, mcp__codex__codex-reply

Result-to-Claim Gate

> 🔒 **Do not wrap this skill in `/loop`, `/schedule`, or `CronCreate`.** It is > verdict-bearing — it judges whether results support a claim. Re-running that > verdict on a wall-clock timer adds no new signal (the verdict changes only > when the *results* change, not when the clock ticks). What you actually want > to schedule is the *external wait that precedes it* — experiments done → then > run this gate **once**. See > [`shared-references/external-cadence.md`](../shared-references/external-cadence.md).

Experiments produce numbers; this gate decides what those numbers *mean*. Collect results from available sources, get a Codex judgment, then auto-route based on the verdict.

Context: $ARGUMENTS

When to Use

• After a set of experiments completes (main results, not just sanity checks)
• Before committing to claims in a paper or review response
• When results are ambiguous and you need an objective second opinion

Workflow

Step 1: Collect Results

Gather experiment data from whatever sources are available in the project:

1. **W&B** (preferred): `wandb.Api().run("<entity>/<project>/<run_id>").history()` — metrics, training curves, comparisons 2. **EXPERIMENT_LOG.md**: full results table with baselines and verdicts 3. **EXPERIMENT_TRACKER.md**: check which experiments are DONE vs still running 4. **Log files**: `ssh server "tail -100 /path/to/training.log"` if no other source 5. **`idea-stage/docs/research_contract.md`** (legacy fallback: `docs/research_contract.md`): intended claims and experiment design

Assemble the key information:

• What experiments were run (method, dataset, config)
• Main metrics and baseline comparisons (deltas)
• The intended claim these experiments were designed to test
• Any known confounds or caveats

Step 1.5: Deterministic evidence pre-check (before spending a Codex call)

For every claim that cites a specific number + a source file, verify the evidence *exists* mechanically — no model call — to catch **hallucinated evidence** before the jury runs (see [`shared-references/evidence-precheck.md`](../shared-references/evidence-precheck.md)).

**1. Build the claims list.** From the cited numbers and their result files, write `[{"id", "value", "source"}, ...]` to `.aris/claims.json` (`source` is the result file/glob relative to the project root; `value` is the cited number or string).

**2. Run the pre-check — this is a real step, not a suggestion.** Execute the block below (resolver per integration-contract §2, **Policy B**: warn-and-skip if the helper is unresolved — never block the audit):

# Policy B = warn-and-skip: nothing here may abort the audit. cd is non-fatal, the
# helper run is explicitly non-blocking, no pipefail-fragile pipe.
cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" 2>/dev/null || true
if [ -z "${ARIS_REPO:-}" ] && [ -f .aris/installed-skills.txt ]; then
    ARIS_REPO=$(awk -F'\t' '$1=="repo_root"{print $2; exit}' .aris/installed-skills.txt 2>/dev/null) || true
fi
if [ -z "${ARIS_REPO:-}" ] && [ -f "$HOME/.aris/repo" ]; then
    ARIS_REPO=$(cat "$HOME/.aris/repo" 2>/dev/null) || true
fi
EVIDENCE_CHECK=".aris/tools/evidence_check.py"
[ -f "$EVIDENCE_CHECK" ] || EVIDENCE_CHECK="tools/evidence_check.py"
[ -f "$EVIDENCE_CHECK" ] || { [ -n "${ARIS_REPO:-}" ] && EVIDENCE_CHECK="$ARIS_REPO/tools/evidence_check.py"; }
[ -f "$EVIDENCE_CHECK" ] || EVIDENCE_CHECK=""

mkdir -p .aris
if [ -n "$EVIDENCE_CHECK" ]; then
    # NB: evidence_check exits 1 when it FINDS hallucinated evidence (value_not_found /
    # path_missing) — that is the useful signal, NOT a failure. So judge success by
    # whether valid JSON was produced, never by exit code. `|| true` keeps set -e calm.
    python3 "$EVIDENCE_CHECK" . --batch .aris/claims.json > .aris/evidence_precheck.json 2>.aris/evidence_precheck.err || true
    if [ -s .aris/evidence_precheck.json ] && python3 -c "import json,sys;json.load(open('.aris/evidence_precheck.json'))" 2>/dev/null; then
        cat .aris/evidence_precheck.json
    else
        echo "WARN: evidence_check produced no valid output (see .aris/evidence_precheck.err);" >&2
        echo "      pre-check skipped (Policy B); the Codex jury still runs." >&2
    fi
else
    echo "WARN: evidence_check.py not resolved at .aris/tools/, tools/, \$ARIS_REPO/tools/, or via ~/.aris/repo." >&2
    echo "      Pre-check skipped (Policy B); the Codex jury still runs. Fix: rerun" >&2
    echo "      bash tools/install_aris.sh, export ARIS_REPO, or copy the helper to tools/." >&2
fi

The output is `{"results": [{id, value, source, status, ...}], "summary": {status: n}}` with `status ∈ {verified, value_not_found, path_missing, unparseable}`.

**3. Act on the statuses.** Any claim returned `value_not_found` or `path_missing` is **hallucinated evidence** — mark it `claim_supported: no` with `integrity_status: evidence_not_found` immediately; do NOT spend a Codex call defending a number that isn't in the data. `unparseable` claims (no usable value/source) just go to the jury normally.

**4. Carry the per-claim status into Step 2.** Feed a small `evidence pre-check: <id> → verified | value_not_found | path_missing | unparseable` table (from `.aris/evidence_precheck.json`) into the Step-2 Codex prompt so the jury knows which claims have real evidence to read. If the pre-check was skipped (helper unresolved), say so in that slot rather than omitting it.

`verified` here means only that the cited evidenc