/autoresearch

name: autoresearch
description: Set up, resume, inspect, or pause a measured experiment loop in Claude Code or Codex. Use when asked to run autoresearch or iteratively optimize a named benchmark. Reviewing this plugin alone does not start an experiment loop.

Autoresearch

Try a scoped change, measure it against a fixed benchmark, keep a supported improvement, and record what happened. Continue within the user's authorization and a finite budget. User interruptions and scope changes take precedence over continuation instructions.

Host and mode

Resolve `AR_SCRIPTS` to the `scripts` directory alongside this skill's actual file. The helpers require Python 3.10+; Bash wrappers and locking support Linux/macOS.

• **Claude Code:** invoke `/autoresearch` for manual installation, or

`/autoresearch:autoresearch` when loaded as a plugin. The four registered Claude hooks handle continuation and compaction. Complete one experiment per turn.

• **Codex:** invoke `$autoresearch` in CLI/IDE, or select this skill in the app.

Use the same protocol with Codex's available read/edit/exec tools. This package's Claude hook registration does not install Codex hooks. For unattended continuation, use `scripts/codex_loop.py` as described in [references/codex.md](references/codex.md). A supervisor turn must complete exactly one experiment and then end. Never nest a supervisor inside a supervised turn.

• **Status:** read state/dashboard/worklog and report; run no experiments and do not

unpause. `python3 "$AR_SCRIPTS/ar_state.py" status` is a read-only status command.

• **Report:** write `autoresearch-report.md` with objective, baseline, best, winning

changes, evidence limitations, failures, and remaining ideas; run no experiments.

• **Off/pause:** create `.autoresearch-off` and stop. During a running benchmark,

cancel safely if possible; preserve partial work and report any unfinished process.

For Claude status/report turns in an active experiment, first create `.autoresearch-inspect` in the experiment workspace, including when this skill was loaded directly. The Stop hook consumes it to let the inspection turn end. For status, this control-file write is the only write; it does not log or run an experiment. Codex status remains read-only and creates no inspection sentinel.

Setup

1. Establish the goal, benchmark command, primary metric and direction, exact files in scope, fixed scorer/data/splits, correctness checks, and budget. Infer these from the request when possible. Authorization to optimize code does not imply permission to push, deploy, contact people, install dependencies, or rent compute. 2. Inspect git status, including untracked files and the index. Use an experiment branch in a clean dedicated worktree when other work is present; preserve that work without stashing or cleaning it away. Require a committed starting point. 3. Read the workload. Write `autoresearch.md` using the outline below and create `experiments/worklog.md`. Prepare `autoresearch.sh` and, when needed, `checks.sh`. Commit only explicitly selected initial code/harness files; keep session artifacts local. Check ignores in the target repository: this plugin's `.gitignore` is not inherited by another project. Add anchored artifact patterns to that project's local git exclude file (locate it with `git rev-parse --git-path info/exclude`): `/autoresearch.jsonl`, `/autoresearch.md`, `/autoresearch-dashboard.md`, `/autoresearch-report.md`, `/autoresearch.ideas.md`, `/experiments/`, `/.autoresearch-off`, `/.autoresearch-inspect`, `/.autoresearch.jsonl.lock`, `/.autoresearch-codex.lock`. Preserve existing excludes. Track the benchmark or explicitly ignore `/autoresearch.sh` if it is session-local. 4. Lock the evaluation definition **before optimization**. Record scorer hashes, dataset/split identity, dependencies, command, seed schedule, and hardware where relevant. Keep benchmark and metric-emitting code outside editable scope. A legitimate scorer fix requires a new segment and a new baseline. 5. Validate the harness with an appropriate trivial/input-independent control. Run the unchanged baseline 3–5 times to estimate noise. Hold evaluation data and folds fixed; vary only training seeds or repeat timing measurements. Record the raw values, mean, and sample standard deviation. A zero measured deviation is not proof of zero uncertainty. 6. Initialize the state with the measured floor, then log the baseline as the first keep (an unchanged baseline need not create an empty commit). Use a finite `maxRuns` (default 200), and a practical wall-time budget. Setup/calibration runs are outside the logged experiment count: record and budget them separately. 7. Continue until a budget, target, user pause, or a real execution failure prevents progress. On failure, preserve evidence, pause, and explain the blocker.

Session outline: `autoresearch.md`

# Autoresearch: <goal>
## Objective
Workload, expected outcome, and what was learned from initial inspection.
## Metrics
Primary name/unit/direction, secondary metrics, measured noise floor and seed values.
## Budget
maxRuns, maxSeconds, optional targetMetric, per-benchmark timeout, setup cost.
## How to Run
Exact benchmark/check commands, working directory, runtime and dependencies.
## Files in Scope
Exact files the experiment may edit.
## Off Limits
Scorer, metric code, data, split/seed definition; record hashes/identity here.
## Constraints
Correctness requirements, resource/network limits, and allowed actions.
## What's Been Tried
Results, failed ideas, insights, and next candidates. Update every 5–10 runs.

Run one experiment

1. Read the current segment, best kept result, session rules, and recent worklog. Check the budget **before launching**. Do not reset a budget to continue. 2. Record starting HEAD and clean status. Choose one hypothesis and its precise edita