challenge-plans
中文文档: README-zh.md
Adversarially review a plan before you execute it — across the AI coding CLIs you already have logged in. No API keys.
challenge-plans orchestrates the subscription CLIs on your machine (Claude Code, Codex, …) to cross-examine a plan and surface the flaws that cause rework — then aggregates only the objections that survive scrutiny into a single verdict. It runs as a CLI, an agent skill, or a CI gate, and slots into the superpowers plan lifecycle: writing-plans → challenge-plans → executing-plans.
Why challenge-plans
- 🔑 No API keys, no per-token charges — it drives the subscription CLIs you're already logged into (Claude Code, Codex). Bring at least one.
- 🧪 Evidence beats headcount — a minority objection with a reproduction can override a majority; correctness isn't decided by voting.
- 🤝 Cross-family verification — an objection only earns hard-gate authority (
✓) when an independent model family reproduces it with line-anchored evidence. Single-model claims stay advisory.
- 🛡️ Guards 7 multi-agent failure modes — vote loss, option anchoring, premature hand-off, majority-over-minority, single-round complacency, false consensus, false convergence (how).
- 🌍 Speaks your language —
--lang zh (or ja, de, …) localizes every finding; one flag, no separate build.
What it does
You don't have to write specs to use this. It works in two modes — review something you've already written, or choose among options. Both hand the content to AIs from different model families to cross-examine it, and keep only the objections backed by evidence that survive another family's check — not one model's hot take.
Review (run) covers four kinds of artifact:
- 📋 Any plan (
--type plan) — a trip, a launch, a hire, a move. Feed it a Kyoto itinerary that crams six stops into Day 2 and books a non-refundable flight now: it tells you which day won't fit and which step you can't undo.
- 📝 A drafted spec or design doc (
--type spec) — before you build. Give it "export user CSV asynchronously": it presses on what "done" means, who signs off, and which edge case is missing — so you don't build the wrong thing from a vague spec.
- 🔧 A code change (
--type diff) — a lightweight review. Pipe in a git diff: it watches for the call site you changed a function but forgot to update, the missing test, the compatibility you quietly broke.
- 🧭 A decision you've already made (
--type decision) — a tech-stack pick, a vendor, a hire. Tell it "we've decided to rewrite the project in TypeScript": it pushes back — did you fairly weigh the cheaper option (just clean up the env)? Is the reason sunk cost? Can you even undo this?
Choose (weigh) is the other mode, for when you're torn between options that are still open — a technical pick or an everyday-life call alike. Picking among Postgres / MongoDB / SQLite, or between three job offers: several AIs vote and rank them, but each model family's weight is capped (so one loud model can't fake a consensus) and the strongest dissent is surfaced — you get a ranking with reasons and an opposition, not a blunt one-word answer.
Quickstart
Easiest — hand it to your agent. Tell it:
Install challenge-plans from https://github.com/hiadrianchen/challenge-plans and run challenge-plans doctor.
It sets up the package and reports which backends are ready.
Or install it yourself (Python ≥ 3.10):
pip install challenge-plans # or: pipx install challenge-plans · uvx challenge-plans doctor
challenge-plans doctor # which CLIs are logged in
# already installed? update with the matching one-liner:
# pip install -U challenge-plans · pipx upgrade challenge-plans
# uvx runs a cached copy — use `uvx challenge-plans@latest …` (or `--refresh`) to get the newest
Bring at least one logged-in CLI — Claude Code (claude) or OpenAI Codex (codex); two different vendors unlock cross-family verification. To use it as an agent skill, drop SKILL.md where your agent discovers skills. (Developing? git clone … && pip install -e ..)
See it work
Here's the "any plan" scenario — one of the four above — on a rough Kyoto-trip plan (examples/plan-sample.md):
$ challenge-plans run examples/plan-sample.md --type plan --sink markdown
# challenge-plans · challenge · verdict: request_changes
- [high✓] Non-refundable flights locked before validating the trip @L10 (irreversibility_or_high_cost, by claude:feasibility)
- [med ] Day 2 packs six sights across the city — likely undoable @L4 (ignored_constraint, by gpt:risk)
- [med ] "A good trip" is never defined, so nothing can be judged @L1 (missing_success_criteria, by claude:goal-alignment)
Each line is a tagged objection — anchored to a line, raised by one reviewer with a single job. The three findings above came from three reviewers:
- Feasibility (can it actually be done?) → caught the non-refundable flight locked in before the plan is even validated.
- Risk (what's most likely to go wrong / is irreversible?) → caught Day 2 cramming six sights across the city with no time budget.
- Goal-alignment (do the steps serve the goal?) → caught that "a good trip" is never defined, so nothing can be judged.
Every objection is tagged from a fixed menu of ways a plan breaks — which keeps findings concrete and de-duplicable. Here's what each one catches, in this trip:
irreversibility_or_high_cost — booking a non-refundable flight before validating the planignored_constraint — six sights in one day, no time or energy budgetmissing_success_criteria — never saying what "a good trip" meansdependency_or_sequencing_gap — a 10:00 train right after "last-minute shopping"unaddressed_risk — going in mid-July with no rain/heat backupunstated_assumption — assuming the famous kaiseki place has a tableno_fallback — no plan B if that restaurant is booked outgoal_misalignment — a "relaxing" trip scheduled dawn to midnight
--profile fast runs one reviewer, standard all three (one round); deep runs the panel for multiple rounds — each round is shown what's already been found and hunts for what the last one missed — and stops when a round surfaces nothing new (convergence).
Usage
As a skill, you don't memorize flags — just ask your agent. Say "review this plan with challenge-plans", "is this spec ready to build?", or even "how do I use challenge-plans?" — it picks the mode and command for you and brings back the surviving objections.
Running it directly? Here's the map:
| I want to… | Run |
|---|
| See which backends are ready | challenge-plans doctor |
| Review any plan | challenge-plans run trip.md --type plan --sink markdown |
| Review a spec before building | challenge-plans run spec.md --type spec --sink markdown |
| Review a code change | git diff > c.diff && challenge-plans run c.diff --type diff |
| Audit a decision already made | challenge-plans run decision.md --type decision --sink markdown |
| Choose among options still open | challenge-plans weigh options.yaml --sink markdown |
| Get findings in Chinese | add --lang zh |
| Use it as a CI gate | add --enforce (request_changes / inconclusive / schema_invalid exit non-zero; discuss/approve exit 0) |
| Use it as a hard gate | add --strict (only a clean approve passes; discuss and approve_with_unverified_timeouts also fail) |
| Keep an audit trail | add --save runs/ (writes a timestamped JSON record — tool version + full manifest — per run) |
decision vs plan vs weigh — which one? They sit at different points in a choice's life. weigh is before you've chosen: ≥2 open options, it ranks them. --type decision is after you've chosen one: it stress-tests that single call (did you skip an alternative, is the evidence real, can you undo it). --type plan is the steps you'll take once chosen: it checks whether they can actually be executed. Same trip — weigh picks fly-vs-train, decision audits "we're flying," plan reviews the day-by-day itinerary.
--profile fast|standard|deep trades speed for depth. [sev✓] = an independent model family reproduced it with line-anchored evidence (cross-model confirmation — not a mechanically-run test), so it may hard-gate; [sev?] = unconfirmed, advisory. Ready-to-run samples live in examples/. Not pip-installed? Prefix with PYTHONPATH=src python3 -m challenge_plans.cli ….
Output in your language
The codebase is English, but reviewers can answer in any language — add --lang:
challenge-plans run plan.md --type plan --lang zh # findings in Chinese
challenge-plans weigh options.yaml --lang ja # deliberation in Japanese
It switches only the human-readable prose; JSON keys, enum values, and line anchors stay verbatim, so parsing and CI gates are unaffected (equivalent to setting CHALLENGE_PLANS_LANG). Your agent can pass --lang <user-language> to localize the whole review.
How it works
Multiple persona/CLI challengers steelman then attack the artifact; a cross-family Verifier must reproduce a high/critical objection with line-anchored evidence before it can hard-gate; findings are de-duplicated and resolved into one 6-state verdict — with an incomplete panel never passing as approve.
Why a tool, and not a one-file prompt / skill.md / MCP? Because the hard part isn't asking a model to "review adversarially" — it's the guarantees around that, which are control-flow, not wording: a different vendor must reproduce a finding before it can gate (not the same model judging itself); the verdict is computed in code from surviving evidence, not asserted by a model; a truncated/timed-out reply is caught instead of silently passing; and it rides your subscription instead of billing per token. In one line — the prompts produce evidence; the Python produces the verdict. The module-by-module map (what each of the 9 scripts does and why), the two modes, the deliberation flow, and the 7 failure modes are in docs/how-it-works.md. It also composes with superpowers and grill-me — see there.
Backends — what you need
Drives whatever subscription coding CLI you already have logged in — Claude Code (claude), OpenAI Codex (codex), or Kimi Code (kimi), via their headless mode, on your existing subscription. No API keys, no per-token charges from this tool. It needs at least one logged-in CLI; challenge-plans doctor names each backend's state, whether it's in your default panel, and the exact fix (install, or log in).
How much you get depends on how many different vendors you have logged in:
| You have | What runs | The catch |
|---|
| One (claude or codex) | Multiple personas attack from different angles on your one subscription — still far more than asking your model once. | Findings stay advisory: nothing model-found can hard-gate, the verdict is capped at discuss (a clean run won't reach approve), and you get a low_diversity warning. The cross-family [sev✓] confirmation is off. A mechanical --verify check (tests/lint failing) can still hard-gate, since that's objective evidence, not a model vote. |
| Two (claude and codex) | Everything above, plus a cross-family Verifier: one vendor's model must independently reproduce a high/critical objection with line-anchored evidence before it can hard-gate. | This is the headline — [sev✓] confirmation, real request_changes/approve verdicts. |
So with one subscription it's a souped-up single-model review; the cross-family verification that makes a finding hard-gate needs a second, different vendor — either another builtin CLI, or a backend you bring yourself.
Which families run — the default panel
Having a family logged in doesn't mean it runs on every review. Supported ≠ on-by-default: the default panel is the verified low-friction pair claude + gpt (the codex CLI's family is gpt), and every other detected family — kimi, a BYO/gateway backend — is opt-in. That's deliberate: subscription quotas differ wildly (an entry Kimi tier has a small window; a gateway may bill per token), so a scarce family is never spent unless you ask for it.
challenge-plans doctor # each ready family is tagged `· default panel` or `· opt-in`
challenge-plans run plan.md --families claude,kimi # this run only: exactly these families
challenge-plans config families claude,gpt,kimi # set a persistent default panel
challenge-plans config show # print the resolved default + config path
--families wins for a single run; a persistent default lives in ~/.config/challenge-plans/config.yaml (config families …, or edit by hand); with neither, the built-in claude + gpt default applies.--families is strict: a name that isn't usable right now is a hard error, not a silent drop (so a typo can't quietly weaken your panel). An implicit default that collapses to one family says so on stderr rather than degrading silently.- The config stores family names only — never tokens or endpoints.
- Kimi is a verified builtin, not a BYO: on a logged-in Kimi membership its family is confirmed from the CLI's own provider config and pinned per run, so it earns the builtin
[sev✓] (unlike a user_declared BYO). It's just opt-in by default like any non-core family. If kimi is pointed at a non-Kimi provider its family can't be verified, so it's withheld rather than mislabelled.
Bring your own backend (BYO)
You can register extra backends that speak the Anthropic-compatible API (GLM, Kimi, DeepSeek, a local proxy, …) through environment variables — challenge-plans drives them with the same claude CLI binary, pointed at your endpoint:
export CP_BYO_1_BASE_URL="https://open.bigmodel.cn/api/anthropic" # your endpoint (required)
export CP_BYO_1_FAMILY="glm" # the model family YOU declare it to be (required)
export CP_BYO_1_TOKEN="…" # endpoint API token (required — see the hygiene note)
export CP_BYO_1_MODEL="glm-4.7" # optional model override
# CP_BYO_2_* / CP_BYO_3_* … register more backends
challenge-plans doctor # → byo-cli-1 (glm, user-declared family): ready
Three safety/honesty rules are built in, not optional:
- The declared family is your word, not something the tool verifies. It cannot see what actually serves your endpoint, so BYO families are labeled
user_declared throughout the manifest, a confirmation minted through one renders as ✓(user-declared family) instead of the builtin ✓, and diversity that depends on it carries a diversity_relies_on_user_declared_family warning. Point _FAMILY=glm at what is really another Claude proxy and you are silently polluting your own review — the labels exist so a reader can tell asserted diversity from the verified builtin claude+gpt pair.
- Your token stays out of every output. It is injected only into that backend's own subprocess env, and never appears in logs, manifests,
--save provenance records, or doctor output (base URLs are excluded from outputs too, since some providers embed keys in them). Still, prefer a secrets manager or a leading-space export over pasting tokens into your shell history.
- The subscription
claude backend is hard-isolated from BYO config. The default adapter strips CP_BYO_* and ANTHROPIC_* from its subprocess env, so your Claude subscription credentials can never be redirected to a third-party endpoint. A half-configured backend (e.g. missing _TOKEN) is skipped with a warning instead of ever falling back to subscription auth.
Status
v1 — usable. Both modes work end-to-end, validated against real plans/specs, pinned by a pytest suite, and hardened across multiple cross-agent adversarial-review rounds (the README itself included). Known boundaries are listed in docs/how-it-works.md.
Contributing
Issues and PRs welcome — see CONTRIBUTING.md. The project is dogfooded: review your own change with challenge-plans run <change>.diff --type diff before opening a PR.
License
Apache-2.0.
