/ds-baseline

name: ds-baseline
description: Use when a quest needs to attach, import, reproduce, repair, verify, compare, or publish a baseline and its metrics.
skill_role: stage
license: MIT
metadata:
  author: ResearAI/DeepScientist
  version: "1.0.0"

Baseline

This skill establishes the reference system the quest will compare against. The target is one trustworthy baseline line, not an endless reproduction diary.

Interaction discipline

• Follow the shared interaction contract injected by the system prompt.
• Keep ordinary setup and debugging updates concise.
• Use richer milestone updates only when the baseline becomes trusted, caveated, blocked, waived, or route-changing.
• Hard execution rule: every terminal command in this stage must go through `bash_exec`; do not use any other terminal path for setup, reproduction, monitoring, verification, Git, Python, package-manager, or file-inspection commands.
• Prefer `bash_exec` for setup, reproduction, monitoring, and verification commands so the baseline line stays durable and auditable.

Tool discipline

• **Do not use native `shell_command` / `command_execution` in this skill.**
• **All shell, CLI, Python, bash, node, git, npm, uv, and environment work must go through `bash_exec(...)`.**
• **For git work inside the current quest repository or worktree, prefer `artifact.git(...)` before raw shell git commands.**
• **If a generic git smoke test is needed outside the quest repo, use `bash_exec(...)` in an isolated scratch repository.**

Non-negotiable rules

• no fabricated metrics, logs, run status, or success claims
• do not skip baseline steps or silently simplify the route when that would change trust or comparability
• do not claim a baseline is ready before verification is complete
• do not infer missing commands, scripts, or parameters when the uncertainty could change the result
• any unavoidable guess must be written down explicitly with expected impact
• use web search for discovering papers or repos, but use `artifact.arxiv(paper_id=..., full_text=False)` for actually reading a source arXiv paper when it exists
• set `full_text=True` only when the short form is insufficient
• for Python baselines, environment setup should be standardized around `uv`

Stage purpose

The baseline stage should produce a usable reference point through one of four routes:

1. attach an existing reusable baseline 2. import a reusable baseline package 3. reproduce a baseline from source 4. repair a broken or stale baseline

Keep the classic control flow:

1. analysis 2. setup 3. execution 4. verification

These are control gates, not paperwork walls.

Quick workflow

1. Read the source paper and source repo first, or record exactly what is missing and why. 2. Choose the lightest trustworthy route: attach, import, reproduce, or repair. 3. Start with the fast path whenever the current baseline object, command path, and acceptance target are already clear enough to validate cheaply. 4. Before substantial baseline setup, code edits, or a real baseline run, create `PLAN.md` and `CHECKLIST.md`; short-form files are enough for simple fast-path work. 5. Keep one dominant phase visible: analysis -> setup -> execution -> verification. 6. Prefer one clean implementation pass, one smoke test, and then one normal baseline run. 7. Retry only when smoke, verification, or runtime evidence shows a concrete failure or incompatibility. 8. Close the stage by confirming or waiving the gate, then hand off with a concise `1-2` sentence summary of trust status and next anchor.

Fast-path first

Default to the lightest baseline path that can still establish a trustworthy comparison. Default to a fast path when it can establish trust with less work.

Fast path is the default when any of the following is true:

• `requested_baseline_ref` or `confirmed_baseline_ref` already points to the active baseline object
• the route is clearly `attach` or `import`
• the repo entrypoint, dataset or split, and metric contract are already concrete enough to validate cheaply
• reproduction requires no meaningful code changes and the main uncertainty is only whether the command still runs

Fast path means:

• do not restart broad baseline discovery by default
• do not front-load a full codebase audit when the entrypoint is already concrete
• use a minimal `PLAN.md`, a minimal `CHECKLIST.md`, one bounded smoke test when needed, and then one real validation or run
• default to reuse-and-verify when runtime already attached a concrete baseline

Escalate from fast path to fuller audit only when:

• the paper and repo disagree materially
• the real run or eval entrypoint is unclear
• code changes are likely required
• the contract spans multiple metrics, datasets, subtasks, or splits that still need interpretation
• the same failure class reappears after one documented autonomous fix
• the quest is trying to publish a reusable global baseline rather than only clear the current gate

Use when

• no credible baseline exists yet
• the current baseline is unverified or stale
• the user already has a baseline package that should be attached or imported
• a reproduction failed earlier and now needs repair
• the quest resumed and the baseline trust state is unclear

Do not use when

• the quest already has a verified active baseline and the next move is ideation or execution
• the user explicitly waived the baseline gate and that waiver is durably recorded

Stage gate

Do not proceed to comparison-heavy downstream work unless one of the following is durably true:

• a baseline has been attached and accepted
• a baseline has been imported and accepted
• a baseline reproduction has completed and been verified
• an explicit waiver decision exists with a clear reason

Operationally:

• call `artifact.confirm_baseline(...)` once the accepted baseline root and trusted comparison contract are clear
• call `artifact.waive_baseline(...)` when the quest must continue without a baseline
• at