autocontext

autocontext is a harness for agent improvement. Give it a goal, it runs the task against evaluation, keeps the useful lessons, discards dead ends, and leaves traces, reports, playbooks, datasets, and optional local-model training artifacts for the next run.

Docs: autocontext.ai/docs · quickstart · CLI reference · changelog

Install

The PyPI package is autocontext; the CLI is autoctx. The npm package is autoctx (not the unrelated autocontext npm package). Provider variables live in .env.example. The npm CLI and TUI require Node.js 22.19.0 or newer; contributors should use the version pinned in ts/.nvmrc.

30-Second Run

Pi is the lowest-friction provider because it uses your local agent auth:

AUTOCONTEXT_AGENT_PROVIDER=pi \
AUTOCONTEXT_PI_COMMAND=pi \
autoctx solve "improve customer-support replies for billing disputes" --iterations 3

Use AUTOCONTEXT_AGENT_PROVIDER=anthropic, openai-compatible, openrouter, claude-cli, codex, pi-rpc, or another provider when you need that runtime. See agent integration for the full matrix.

Running it on your own GPU instead? Self-hosted models covers the whole loop on vLLM, Ollama, or any OpenAI-compatible endpoint — including what each role actually resolves to, and why constrained output matters more on open weights. Self-hosted endpoints can additionally declare AUTOCONTEXT_PROVIDER_HOSTING=local and a fast, mid_tier, or frontier AUTOCONTEXT_PROVIDER_CAPABILITY; role-specific endpoints use matching <ROLE>_PROVIDER_* declarations.

Prime remote execution also supports opt-in accelerator requests with explicit type/count, immutable-image, region, and telemetry capability validation. It fails before provider creation when the configured pool cannot satisfy the request and never downgrades accelerator work to CPU; see remote execution sessions. Shipped Prime generation and campaign paths persist a durable pre-dispatch claim plus the complete result/ledger projection before returning paid work; restart never treats an unresolved or already committed request as permission to provision another sandbox.

Agent Entry Points

• Pi: install pi-autocontext, then ask Pi to solve, judge, improve, list, or inspect runs through the packaged skill.
• MCP clients: run autoctx serve mcp or bunx autoctx serve mcp and expose the tools to Claude Code, Cursor, or another MCP client.
• Hermes: export the CLI-first skill with uv run autoctx hermes export-skill --with-references --json.

Full setup: autocontext/docs/agent-integration.md.

What A Run Leaves Behind

runs/<run_id>/
├── trace.jsonl
├── generations/<n>/{strategy.json,analysis.md,score.json}
├── report.md
└── artifacts/

knowledge/<scenario>/
├── playbook.md
├── hints.md
├── tools/
└── context_bundles/{bundles,candidates,promotions,active.json}

Everything is filesystem-first: inspect it, diff it, replay it, export it, or feed it into training. Kernel campaigns extend that contract with exact provider-generation receipts, bounded paid-call accounting, content-addressed lineage, and safe stop/status/resume. Coach and architect context changes are stored as immutable candidates and are not served until matched candidate/incumbent trials confirm them. The live serving boundary can additionally require a cancellable independent audit and a durable campaign-wide false-promotion budget; exact causal credit is accepted only from verified single-component manifest additions. See context bundles and outcome-gated promotion. Python also provides an opt-in executable-skill bridge for verified schema-migration proposals in Docker, with explicit eligibility and promotion-gated serving. An opt-in skill/model router adds verified fallback and explicit abstention under one request budget. The schema-migration reuse study compares baseline, textual context, executable reuse and cheaper-model controls under frozen paired evaluation; fixture runs are infrastructure evidence only. Controlled component trials feed ablation-backed attribution, so prompt selection can demote low-value context without presenting edit-size correlation as causal.

Python kernel evolution can also compose bounded studies across variable-shape matmul, fused elementwise/reduction, and causal-attention families. Each family retains independent primary/confirmation evidence and per-case floors; cross-shape, cross-hardware, and cross-family trials distinguish portable, partially transferring, specialist, and plateau outcomes without an aggregate score hiding a failed workload. See the kernel evolution guide.

Core Surfaces

Running bare autoctx shows the concise paved-road workflow. Use autoctx --help --all in the npm CLI or autoctx commands --all in the Python CLI for the full catalog. --iterations is the primary iteration flag; --gens is a compatibility alias. autoctx --version --json reports the package version and runtime (python or typescript).

Python owns the full control-plane package; TypeScript owns several operator-facing surfaces, the TUI, and Node runtime adapters. Start with autocontext/README.md or ts/README.md.

HTTP and WebSocket control planes bind to loopback by default. A non-loopback bind fails closed unless AUTOCONTEXT_SERVER_TOKEN contains a random value of at least 32 characters. Send it as an Authorization: Bearer value; browser WebSocket clients use the autocontext.bearer.<base64url-token> subprotocol. Tokens in endpoint URLs are rejected. See the persistent-host security model before exposing a server beyond one trusted operator.

What's New in 0.19.0

• Safer run re-entry: autoctx resume continues a run with its stored scenario and target, completed runs stay unchanged, and re-entering under a different provider or executor mode is refused unless resume gets --allow-runtime-change. Agent-task runs, A/B tests and TypeScript runs refuse run ids that already exist.
• Reliable shared databases: several processes can start against one new SQLite database without migration-ledger or WAL lock failures, and cross-runtime migrations keep runs.minimum_generations, re-adding it where an earlier migration dropped it.
• Correct artifact paths: Python run artifacts no longer land at doubled paths under relative artifact roots, and artifact_write hooks in either runtime cannot redirect a write out of its managed root through a symlink.
• Opt-in executable skills (Python): for a schema-migration pilot, context bundles can carry Docker-isolated executable skills routed with verified model fallback, and a complete routed policy is promoted only through a fail-closed quality and cost gate. Nothing is enabled automatically.
• Evaluator provenance and human labels (Python): judge epochs bind an immutable serving specification that includes calibration examples, and autoctx labels collects human-only judge labels with random audits and protected holdout splits.
• Measured reuse studies: archived GridCTF policy-reuse and schema-migration reuse studies report quality, fallback and resource costs. GridCTF dollar costs are unknown, the schema-migration study records model-credit charges but not full lifecycle cost, and neither study authorizes production promotion.

npm 0.17.3 minimum iteration floors

autoctx@0.17.3 advertises minimum_iterations_v1. Structured task contracts may set minimumIterations, interactive start_run commands may override it