/ds-experiment

name: ds-experiment
description: Use when a quest is ready for a concrete implementation pass or a main experiment run tied to a selected idea and an accepted baseline.
skill_role: stage
license: MIT
metadata:
  author: ResearAI/DeepScientist
  version: "1.0.0"

Experiment

Use this skill for the main evidence-producing runs of the quest.

Interaction discipline

• Follow the shared interaction contract injected by the system prompt.
• For ordinary active work, prefer a concise progress update once work has crossed roughly 6 tool calls with a human-meaningful delta, and do not drift beyond roughly 12 tool calls or about 8 minutes without a user-visible update.
• Keep ordinary subtask completions concise. When a main experiment actually finishes or reaches a stage-significant checkpoint, upgrade to a richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` report rather than another short progress line.
• That richer experiment-stage milestone report should normally cover: what run finished, the headline result versus baseline or expectation, the main caveat, and the exact recommended next action.
• That richer milestone report is still normally non-blocking. If the next route is already justified locally, continue automatically after reporting rather than idling for acknowledgment.
• If the active communication surface is QQ and QQ milestone media is enabled in config, a completed main experiment may attach one summary PNG to that richer milestone update.
• That PNG should be a connector-facing report chart, not a raw debug plot and not a draft paper figure.
• Do not auto-send every training curve, per-step plot, or intermediate slice image.
• Preferred connector-chart palettes are Morandi-like and restrained:
• `sage-clay`: `#E7E1D6`, `#B7A99A`, `#7F8F84` for the default QQ summary look
• `mist-stone`: `#F3EEE8`, `#D8D1C7`, `#8A9199` for conservative summaries
• `dust-rose`: `#F2E9E6`, `#D8C3BC`, `#B88C8C` for secondary comparisons only
• Connector-facing chart requirements:
• white or near-white background
• low saturation, no neon colors
• one primary accent plus one neutral comparison color whenever possible
• simple legend, light grid, readable labels, and no dashboard clutter
• summarize only the evidence needed for the milestone
• Default chart choice:
• line chart for training / budget / step trends
• bar chart for a small number of categorical end-point comparisons
• point-range chart when uncertainty or seed spread matters
• If the figure encodes ordered magnitude, use a sequential muted palette; if it encodes signed delta around a reference, use a diverging muted palette with a neutral midpoint.
• Avoid rainbow / jet-like colormaps, 3D effects, and over-annotated dashboards.
• If the chart may later be reused in the paper, export a vector copy (`pdf` or `svg`) alongside the connector `png`.
• If the figure matters beyond transient debugging, open `figure-polish/SKILL.md` and follow its render-inspect-revise workflow before treating the image as final.
• If plotting in Python, reuse the fixed Morandi plotting starter from the system prompt rather than inventing a new bright style for each run.
• If the runtime starts an auto-continue turn with no new user message, continue from the current run state, logs, artifacts, and active requirements instead of replaying the previous user turn.
• Progress message templates are references only. Adapt to the actual context and vary wording so messages feel human, respectful, and non-robotic.
• If a threaded user reply arrives, interpret it relative to the latest experiment progress update before assuming the task changed completely.
• Hard execution rule: every terminal command in this stage must go through `bash_exec`; do not use any other terminal path for smoke tests, real runs, Git, Python, package-manager, or file-inspection commands.
• Prefer `bash_exec` for experiment commands so each run gets a durable session id, quest-local log folder, and later `read/list/kill` control.
• For meaningful long-running runs, include the estimated next reply time or next check-in window whenever it is defensible.

Tool discipline

• **Do not use native `shell_command` / `command_execution` in this skill.**
• **All smoke tests, real runs, shell, CLI, Python, bash, node, git, npm, uv, and environment work must go through `bash_exec(...)`.**
• **For git work inside the current quest repository or worktree, prefer `artifact.git(...)` before raw shell git commands.**
• **If a scratch repository or isolated test environment is needed, create and drive it through `bash_exec(...)`, not native shell tools.**

Stage purpose

The experiment stage should turn a selected idea into auditable evidence. It should preserve the strongest old experiment-planning and execution discipline:

• define the run contract before execution
• keep the run comparable to baseline
• capture configs, commands, logs, and metrics
• report both success and failure honestly
• route the next action through an explicit decision

The experiment stage is not just "run code". It is the stage that converts an idea contract into evidence that other stages can trust. It is also the stage that should decide the next route once the measured result exists. Within the user's explicit constraints, maximize valid evidence per unit time and compute. Prefer equivalence-preserving efficiency upgrades first: larger safe batch size, mixed precision, gradient accumulation, dataloader workers, cache reuse, checkpoint resume, precomputed features, and smaller pilots. If a proposed efficiency change alters optimization dynamics, effective budget, or baseline comparability, treat it as a real experiment change and record it as such.

Use `references/evidence-ladder.md` when deciding whether the current package is merely executable, solid enough to carry the main claim, or already in the stage where broader polish is justified.

Completing one main run i