/spawn-ori-eval

name: spawn-ori-eval
description: Spawn Ori as a subprocess to run a throwaway model eval on a pinned harness and model, then relay the results. Use when the user asks which model they should use, wants to compare models, wants to measure whether their agent or prompt does the right thing, wants to catch regressions in agent behavior, or asks how good their current model is. Applies to any codebase in any language. Do not use for plain unit tests that involve no model, and do not use to re-run an eval that already exists (run `ori eval <file>` directly).

Spawn Ori Eval

Ori writes and grades the eval on a pinned harness and model, so the bench is identical for every coding agent. You run Ori, keep the user informed, and relay the result. An eval you write yourself is not reproducible, and a score change must come from the user's agent, not from the environment.

Pins

This section gives the two models. The other sections use the names `<RUN_MODEL>` and `<JUDGE_MODEL>`. Replace each name with the model that this section gives. To change a model, change only the two lines that follow.

• `RUN_MODEL` is `openai/gpt-5.6-terra`. The run uses this model.
• `JUDGE_MODEL` is `openai/gpt-5.6-terra`. The eval judge uses this model.

Steps

Do these in order. One line, one action. Appendix letters point to the detail and run in step order, except the troubleshooting table, which is a lookup and comes last.

1. Create the run directory and derive its path from the repo root (appendix A). 2. Tell the user where the run directory is. 3. Read whatever is already in the directory without changing it, and tell the user what is there and how old it is (appendix A). 4. Ask the user what to do with it, then reuse the existing `steps.txt` and jump to the first step after 4 that is not done, or archive the files and write a fresh `steps.txt` covering step 5 onward, or stop and leave everything untouched. Write the fresh tracker without asking only when the directory holds no run files of its own (appendix A). 5. Run the lookup or install for the `ori` binary yourself (appendix B). 6. If it is still missing, run the `~/.local/bin/ori` fallback yourself, and stop if that fails too. 7. Run `ori auth` yourself, read its output, and branch on the exit status and message: continue when access resolves, stop with login instructions when no credential resolves, and stop with instructions to use 0.3.0 or newer when the command is unknown because the installed Ori predates the headless run support this skill requires (appendix B). 8. Check for `bun` yourself, and stop if it is missing. 9. Read the eval surface yourself, continuing even if the commands error (appendix B). 10. Tell the user where the binary landed, if you installed it. 11. Tell the user what the run will do, from what you read in step 9. 12. Tell the user it takes 10 to 30 minutes and can spend more than the credit on their key. 13. Tell the user they get a scored table and that a question can restart the run. 14. Write the task prompt file (appendix C). 15. Start one run from the repo root and capture its answer and error files (appendix D). 16. Wait for the run to exit, then read the answer file (appendix E). 17. Show the user Ori's narration lines from the answer file in plain language. 18. If the completed answer contains a tagged question anywhere or its assistant text, above the summary line, ends on an untagged question, continue to step 19; relay only the first question, report a broken one-question contract if another appears, and report an untagged question as a violation while still relaying it (appendix E). 19. Print Ori's full context for the first question in an ordinary message, including any table. Do nothing else in this step (appendix E). 20. Update `steps.txt` here, between the context message and the question, recording what was actually printed: quote the table header row that was printed when the answer carried a table, or record that it carried no table (appendix E). 21. Ask the user with your own question UI, preserving the three short options and free-text `Other`. Keep the question to one short sentence and do not restate or summarize the context table (appendix E). 22. Append the question and the user's own reply to the task prompt file, or, when the reply does not answer the question, append nothing, ask the same question again through the question UI, and do this step again on the next reply (appendix E). 23. Restart over the whole prompt file with the next attempt number and return to step 16, but only when step 22 appended an answer (appendix E). 24. Relay the result table, the ship or no-ship decision, and the quoted failures. 25. Relay Ori's cost and timing table in full (appendix F). 26. Add each attempt's own duration and cost from the summary line at the end of its answer file (appendix F). 27. Add one line on the cheaper cost of a re-run. 28. Tell the user where Ori left the temporary workspace and that it is throwaway. 29. Say they can move the eval into their repo if the numbers made them want to keep it.

Rules

These hold for the whole run.

• Never write the eval yourself and never delegate it to your own subagent. Ori's `create-eval` skill runs automatically inside the run.
• Always pass `--model <RUN_MODEL>` and never pass `--harness`. The pin is what makes every coding agent run the same bench. Never substitute another slug and never leave the flag off, because either one hands the run to whatever default the user's install happens to carry.
• Always pass `--prompt-file`. The `-p` flag works but never use it here, because a one-time string cannot carry state across a restart, and a bare positional prompt is rejected outright.
• Run every command in steps 5 to 9 yourself. Installing the binary when it is missing is expected. The credential check is the only setup handoff.
• Never resume and never clear a previous run on your own judgement. The skill is loaded and run inside one session, so a