/adversarial-empirical-review

name: adversarial-empirical-review
description: "N-round adversarial review pipeline for empirical research output — the chain from data to LaTeX tables to a manuscript that cites them. A Claude drafter proposes minimal diffs, a deterministic mechanical battery gates every diff from a clean state with a regression gate, a Codex reviewer files check-backed critiques, and a blind judge panel decides residual disputes. Manual-invoke ONLY: trigger when the user explicitly runs /adversarial-empirical-review or names 'adversarial-empirical-review' / 'adversarial empirical review'. Do NOT auto-trigger on generic 'review my results', 'check my tables', or manuscript-editing requests. For prose-style refinement use style-emulation instead; this skill AUDITS WHETHER THE TABLES ARE CORRECT — that each number in the tables is what the analysis code computes, reproduces from the data, and is internally consistent. It is an empirical + code review: the manuscript is read only to resolve table numbering, and prose is not examined."

Adversarial Empirical Review

Runs a regression-gated, N-round adversarial loop over a project's empirical output. The design is in `docs/2026-06-08-adversarial-empirical-review-design.md` (v2) and the module API in `docs/CONTRACTS.md`. Read the design before operating the skill.

The costly error here is corrupting a result that was already correct, so the whole pipeline is incumbent-preserving: a hard failure-set regression gate, "no clear difference keeps the incumbent" for residuals, and minimization of LLM judgment in favour of mechanical checks.

**The crux is whether the tables are correct.** Auditing an empirical manuscript means verifying that the numbers in its tables are what the analysis code actually computes — that each coefficient, standard error, p-value, N, and R² reproduces from the data and the code and is internally consistent. This is an empirical + code review: the battery, the reviewer, and the drafter are all scoped to **table-number correctness**, the manuscript is consulted only as the spine that maps paper Table numbers to fragment files, and prose is not examined (the prose-scanning catalog checks default to off; research-design and interpretation critique are out of scope, with the consequential-category machinery retained as a fail-safe). The mechanical anchor for this is `builtin.table_source_agreement` (every estout-fragment cell is audited against a `numbers.json` of computed values) together with the reproduction tier and the coef/SE/t, star/p, bounds, and sample-size checks. The crux is **fail-closed**: a manuscript table with no `numbers.json` spec is a `FAIL` and a missing artifact a `CHECK_ERROR`, and the driver will not converge unless every `required_checks` entry (default the crux) is present and passing — otherwise it stops with `crux_unverified`. A manuscript whose tables are hand-transcribed estout fragments decoupled from the regressions (the common case) is reviewed by recomputing the regressions and diffing the cells — supply a `numbers.json` of computed values so the audit is mechanical rather than left to the reviewer's eye.

**Generation-based certification (the path past the parse-and-compare ceiling).** Parse-and-compare has a structural limit: a second numeric token inside one cell, a number in a spanning descriptor row, or content behind an exotic macro is invisible to any LaTeX parser. The durable fix is to GENERATE the fragments from the computed source so displayed == computed by construction: `scripts/tablegen.py` renders estout-style fragments deterministically from `layouts` entries in the same `numbers.json` (the project's pipeline calls `python tablegen.py --numbers ... --tables-dir ...` after the analysis writes the artifact), and `builtin.fragment_regeneration` certifies each laid-out table by re-rendering it and comparing **byte-for-byte** with the on-disk fragment — any drift at all is a `FAIL`, so the parser gaps disappear for generated tables. The parse-and-compare crux keeps running on the same artifact as defense in depth. Mixed mode is supported (generated and hand-written tables side by side; the PASS message names the uncovered tables); `require_all: true` under `battery.builtin["builtin.fragment_regeneration"]` demands a layout for every manuscript table, and projects that adopt generation should add the check to `required_checks` so an unevaluated state blocks convergence exactly like the crux. Keys under `tables` starting with `_` are auxiliary value rows (placeholder sources for composite cells), exempt from the crux's missing-row rule.

**From-raw mode (`snapshot_from_raw: true`).** The strongest re-execution tier: the driver copies the project, **deletes every committed intermediate** (snapshot-partition files, output-partition files, `protected_paths` caches, the numbers artifact — directory globs expanded, deletions confined to the copy), runs the full raw inputs → cleaning → analysis → tables pipeline (`rebuild.from_raw_command`, or `rebuild.from_raw_argv` as an exact argv list — preferred on Windows) twice, and certifies three checks: `builtin.raw_table_identity` (the regenerated LaTeX table files are IDENTICAL to the committed ones, modulo line endings/trailing whitespace), `builtin.raw_numbers_agreement` (numbers artifact within tolerance), and `builtin.raw_determinism` (the two runs agree). Activation is fail-closed: `partitions.raw` plus a from-raw command are required, the raw partition is read-only and provenance-hashed like the snapshot, the identity check is auto-added to `required_checks` so the run cannot converge while the from-raw chain is unverified, and `driver init` halts if any project file matches no partition glob (an unclassified file is a purge-coverage hole). The driver runs all of this inline — no orchestrator action is needed beyond stepping — but expect long battery phases: the full pipeline executes twice per re-execution, bounded by `rebuild.from_raw_timeou