/evolving-with-swarms

name: evolving-with-swarms
description: Improves a program against an automatic scorer by running rounds of cheap Haiku 5.5 swarms, where each proposer suggests one short idea and one writer per swarm turns the ideas into a new program that is scored and kept in an archive, all metered to a dollar cap through headless claude -p. Use for "evolve this program against my evaluator", "optimize this function's score cheaply", "OpenEvolve-style search on a budget", "AlphaEvolve-style construction search with Haiku", "run a swarm to maximize this score", "search for a better heuristic or construction", or any program whose quality a script can score, when the budget is small. Ships the driver, a gateway with a dollar cap and a usage-window pause, a sandbox hook for model-written code, an example problem and an offline selftest; runs from Claude Code on the web or the desktop.
metadata:
  version: 0.1.1

Evolving with swarms

Each round, `scripts/evolve.py` runs K swarms in parallel. A swarm is N proposer calls, each reading one parent program and one focus x thinking-style prompt and replying with an idea of at most 45 words, then one writer call that turns the ideas and the parents into a complete program. The program is scored by your evaluator as soon as it exists and joins the archive the next round draws parents from: refine one of the best two, combine three of the best ten, explore two from anywhere, start fresh from the initial program. Every model call goes through headless `claude -p`, is priced at list rates from its `modelUsage`, and counts against a dollar cap.

The design is a text-only version of SwarmEvolve (Jiang and Karniadakis, arXiv 2611.08483), whose agents think inside an open model's activations, which an API model does not expose.

When it pays

Measured 2026-11-09 against stock OpenEvolve on Sonnet 5.5, three seeds per arm, three problems ([references/evidence-2026-11.md](references/evidence-2026-11.md)):

| | swarm (defaults) | Sonnet loop, best effort per problem | |---|---|---| | $ to 99% of reference, circle packing | $0.013–0.019 | $0.32–0.44 | | $ to 99%, hexagon packing | $0.035–0.060 | $0.53–0.58 | | $ to 99%, Heilbronn triangle | $0.09–0.50 | $0.77–1.02 | | best at $0.25 | above every paired Sonnet seed on all three | | | final score at $3 | swarm stops at 60 min, about $1 | the same finals, in 7–42 minutes |

Use the swarm when the budget is small or the goal is "good, cheaply". When the last 0.1% matters, the money is there, and wall-clock time counts, a Sonnet loop at the problem's best effort level gets to the same place faster.

When NOT to use this skill

| situation | use instead | |---|---| | hundreds of failing tests to drive to green | `swarming-to-green` | | choosing a model or effort for one subagent or a cascade | `agent-routing` | | many independent tasks with checkable outputs, not one program to improve | `dispatching-subagents` (headless runner) | | quality only a person or a judge model can rate | no scorer, no search: write the scorer first, or use a judged loop | | claude.ai chat, where there is no `claude` binary | `orchestrating-chat-agents` |

Abandon a run mid-way when:

• `status` shows `rejected_above_max` climbing or a best score above anything the

problem allows: the scorer has a hole and the search is now optimising the hole;

• the best score is flat for ten rounds and `rounds.jsonl` shows every refine swarm

parenting the same program: the archive has converged, and more rounds buy nothing;

• `last_quota.seven_day` passes about 0.40: the gateway stops at 0.45 anyway.

Earned exceptions

| rule | banned when | earned when | |---|---|---| | writer at effort medium | — | effort low on a problem already known to be easy (circle packing: low reached 0.999 cheaper) | | N=8 proposers per swarm | the problem is new and N=0 has not been tried | an N=0 control on this problem lost to N=8 (on circle packing the writer alone nearly tied) | | no sandbox prefix | the evaluator imports model-written code on a machine with secrets or network | a throwaway container with no credentials, and the user agreed |

Setup

1. **Make a problem directory.** Copy `assets/example/` and replace its four files:

• `problem.md`: the task in plain words, including the reference value if known;
• `initial_program.py`: a valid, weak starting program;
• `evaluator.py`: `evaluate(program_path) -> dict` of numbers and strings, with the

score under `score_key` (default `combined_score`) and an `error` key on failure;

• `config.toml`: entry function name, the interface text the models see, budgets.

2. **Score the initial program by hand** and read the dict:

   python3 -c "import importlib.util as u; s=u.spec_from_file_location('e','evaluator.py'); e=u.module_from_spec(s); s.loader.exec_module(e); print(e.evaluate('initial_program.py'))"

A score of 0 with no `error`, or a missing `score_key`, means the contract is wrong; fix it before spending anything. 3. **Set `score_max`** from a bound you can prove (a known optimum, an area or volume bound), slightly above it. Scores above it count as 0 and are flagged. Leave it unset only when no bound exists, and then re-check the best program independently. 4. **Choose the sandbox.** The evaluator imports and runs model-written code.

• Claude Code on the web with the claude-workspace hub: the hub jail.

     [sandbox]
     prefix = ["sh", "/home/user/claude-workspace/scripts/jail.sh", "-C", "{run_dir}", "--"]
     chown_uid = 65534

and put `--run-dir` under `/tmp/jail/<name>/` (mkdir and `chmod 755` it first). Launch through a script file: the hub's jail_route guard refuses a Bash command that names `/tmp/jail/` next to `python3`.

• Desktop with Docker: `prefix = ["docker", "run", "--rm", "--network=none", "-v", "{run_dir}:{run_dir}", "python:3.12-slim"]`

(add numpy or what the evaluator needs to the image).

• N