Development
Skill
/prompt-optimizer
A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results.
Claude Code
$ npx -y skills add haabe/mycelium --skill prompt-optimizer --agent claude-code
Codex
$ npx skills add haabe/mycelium --skill prompt-optimizer -a codex -y
Cursor
$ npx skills add haabe/mycelium --skill prompt-optimizer -a cursor -y
Gemini CLI
$ npx skills add haabe/mycelium --skill prompt-optimizer -a gemini-cli -y
GitHub Copilot
$ npx skills add haabe/mycelium --skill prompt-optimizer -a github-copilot -y
OpenCode
$ npx skills add haabe/mycelium --skill prompt-optimizer -a opencode -y
Windsurf
$ npx skills add haabe/mycelium --skill prompt-optimizer -a windsurf -y
SKILL.md
prompt-optimizer.SKILL.mdname: prompt-optimizer
description: "A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results."
metadata:
instruction_budget: "17"
framework_dependency: "mycelium"
framework_dependency_note: "This skill is designed to run within the Mycelium framework (https://github.com/haabe/mycelium). Standalone use will skip the canvas state, theory gates, and harness behavior the skill assumes. Install: /plugin install mycelium@haabe-mycelium."
Prompt Optimizer
Systematically improve Mycelium instructions through measurement. Adapted from n-trax.
Commands
`baseline` -- Capture current performance
1. Run `/mycelium:eval-runner run-split optimization` — record as optimization scores 2. Run `/mycelium:eval-runner run-split holdout` — record as holdout scores 3. Record both to `.claude/optimization/baseline.json`: timestamp, CLAUDE.md hash, optimization metrics, holdout metrics, overall and per-category metrics
`test <variant>` -- Test a variant
1. Read variant from `.claude/optimization/variants/<variant>.md` 2. Apply the CLAUDE.md changes described 3. Run `/mycelium:eval-runner run-split optimization` — this is the hill-climbing signal 4. Run `/mycelium:eval-runner run-split holdout` — this validates generalization 5. Store results in `.claude/optimization/results/<variant>.json` 6. Compare against baseline. Flag **overfitting** if optimization improves but holdout degrades. 7. Do NOT auto-revert -- let user decide
`report` -- Compare all variants
Generate comparison table with split-aware columns:
| Variant | Opt Pass Rate | Holdout Pass Rate | Delta Opt | Delta Holdout | Overfit? | Decision |
Flag `Overfit? = YES` when optimization delta is positive but holdout delta is negative.
`exemplar <eval-name>` -- Capture winning trajectory
After a clean eval win (1 iteration, fast), save the approach to `.claude/optimization/exemplars/`.
Workflow
1. Capture baseline 2. Hypothesize an instruction improvement 3. Document in variants/ directory 4. Test the variant 5. Compare via report 6. Keep or revert based on data 7. Capture exemplars from clean wins
Read more
name: prompt-optimizer description: "A/B test CLAUDE.md instruction changes against eval benchmarks. Capture baselines, test variants, compare results." metadata: instruction_budget: "17" framework_dependency: "mycelium" framework_dependency_note: "This skill is designed to run within the Mycelium framework (https://github.com/haabe/mycelium). Standalone use will skip the canvas state, theory gates, and harness behavior the skill assumes. Install: /plugin install mycelium@haabe-mycelium."
Prompt Optimizer
Systematically improve Mycelium instructions through measurement. Adapted from n-trax.
Commands
`baseline` -- Capture current performance
1. Run `/mycelium:eval-runner run-split optimization` — record as optimization scores 2. Run `/mycelium:eval-runner run-split holdout` — record as holdout scores 3. Record both to `.claude/optimization/baseline.json`: timestamp, CLAUDE.md hash, optimization metrics, holdout metrics, overall and per-category metrics
`test <variant>` -- Test a variant
1. Read variant from `.claude/optimization/variants/<variant>.md` 2. Apply the CLAUDE.md changes described 3. Run `/mycelium:eval-runner run-split optimization` — this is the hill-climbing signal 4. Run `/mycelium:eval-runner run-split holdout` — this validates generalization 5. Store results in `.claude/optimization/results/<variant>.json` 6. Compare against baseline. Flag **overfitting** if optimization improves but holdout degrades. 7. Do NOT auto-revert -- let user decide
`report` -- Compare all variants
Generate comparison table with split-aware columns:
| Variant | Opt Pass Rate | Holdout Pass Rate | Delta Opt | Delta Holdout | Overfit? | Decision |
Flag `Overfit? = YES` when optimization delta is positive but holdout delta is negative.
`exemplar <eval-name>` -- Capture winning trajectory
After a clean eval win (1 iteration, fast), save the approach to `.claude/optimization/exemplars/`.
Workflow
1. Capture baseline 2. Hypothesize an instruction improvement 3. Document in variants/ directory 4. Test the variant 5. Compare via report 6. Keep or revert based on data 7. Capture exemplars from clean wins
Ships withmycelium
Mycelium won't make your idea good. It keeps you and your agent honest about whether it is, from first idea to market. Built on Claude Code, where the gates are structural. The files and skills port to opencode, Codex and Cursor. Outcome over output.
5263 skills
Stars on GitHub
Momentum
- stars in total
- 52
Other skills on mycelium.
More skills from other plugins.
Auto-invoked
brainstorming
You MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or…
@obraInstall Skillask-matt
Ask which skill or flow fits your situation. A router over the skills in this repo.
@mattpocockInstall SkillAuto-invoked
Questions about prompt-optimizer.
- Which coding agents can use prompt-optimizer?
- prompt-optimizer installs in Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode and Windsurf with the skills CLI, using each agent's flag shown on this page. Flowy's auto-invocation, which fires a skill when your prompt matches, runs in Claude Code only.