/experiment-agent

name: experiment-agent
description: "Experiment executor and monitor for academic research. 2-agent system covering code experiments (ML training, statistical analysis, ETL, simulation) and human studies (surveys, field studies, interviews). 4 modes: run (execute + monitor code), manage (track human studies), validate (statistical interpretation + reproducibility verification), plan (Socratic experiment design). Use when a researcher wants to run or monitor a research experiment, manage or resume a human-subject study, check the statistics or reproducibility of research results, or design an experiment or study, in English or Chinese (e.g., 跑實驗、管理研究、驗證結果、規劃實驗)."
metadata:
  version: "1.2.0"
  last_updated: "2026-09-25"
  author: "Cheng-I Wu"
  license: "CC-BY-NC 4.0"
  status: active
  related_skills:
    - academic-pipeline
    - deep-research
    - academic-paper
    - academic-paper-reviewer

Experiment Agent v1.2.0 — Experiment Executor and Monitor

Execute, monitor, interpret, and verify experiments for academic research. Works independently or as an optional bridge between ARS Stage 1 (RESEARCH) and Stage 2 (WRITE).

**Role**: Executor + Monitor. This skill does NOT judge whether results are good for a paper (that is the reviewer's job). It ensures experiments complete successfully, interprets statistical output, and verifies reproducibility.

Quick Start

**Run a code experiment:**

Run my training script: python train.py --epochs 50 --output results/

**Manage a human study:**

Help me manage my survey study — I need 200 responses by May 30

**Validate results:**

Validate these regression results: results/analysis_output.csv

**Plan an experiment:**

Help me design an experiment to test whether AI tools improve QA officer productivity

---

Trigger Keywords

**English**: run experiment, execute code, train model, benchmark, analyze data, manage study, track participants, field study, survey, validate results, check statistics, reproduce, re-run, plan experiment, design study, what should I test

**Chinese**: 跑實驗, 執行程式, 訓練模型, 基準測試, 分析資料, 管理研究, 追蹤參與者, 田野研究, 問卷, 驗證結果, 檢查統計, 重現, 規劃實驗, 設計研究

---

Modes

| Mode | Purpose | Agent | Spectrum | |------|---------|-------|----------| | `run` | Execute code experiments + real-time monitoring | code_runner_agent | Fidelity | | `manage` | Manage human study workflow + progress tracking | study_manager_agent | Balanced | | `validate` | Statistical interpretation + reproducibility verification | SKILL.md (stats) + code_runner_agent (re-run) | Fidelity | | `plan` | Socratic dialogue to design experiments | SKILL.md direct | Originality |

Mode Selection

| User Signal | Mode | |-------------|------| | Has a script/command to run | `run` | | Running a survey, interview, field study, lab experiment | `manage` | | Has results, wants to check numbers or reproduce | `validate` | | Wants to figure out what experiment to do | `plan` | | Ambiguous | Ask: "Are you running code or managing a human study?" |

---

Routing

1. Detect intent from user's first message using trigger keywords 2. Code execution keywords → dispatch `code_runner_agent` (run mode) 3. Human study keywords → dispatch `study_manager_agent` (manage mode)

• **Session resume**: If the user's first message in a session matches `resume <argument>` (where argument is a study_id slug or a path to a state.md file), OR if any later turn matches `resume <argument>` and no artifact write has occurred this session, route to study_manager_agent's RESUME entry path. The agent will read the artifact, validate, and prompt user confirmation before resuming the study at its last known phase.

4. Validation keywords → enter validate mode (handled inline, see below) 5. Design keywords → enter plan mode (handled inline, see below)

Dispatching or delegating to an agent in this skill means reading its file under `agents/` and following it yourself in this conversation: both agents need the user mid-task (confirming a command, choosing what to do about an anomaly, answering protocol questions one at a time).

Runtime Requirements

Most modes work with any LLM runtime that supports prompt + reasoning.

**Session resume in `manage` mode** additionally requires the runtime to provide Read, Write, and Edit tool access to the local filesystem. Claude Code provides these. Runtimes that surface only chat I/O can use PLAN and ETHICS in-session, but study state will not persist across restarts, the `resume <study_id>` command will be unavailable, and a study cannot move to TRACK.

**The study state checker** (`scripts/check_study_state.py`) validates `manage` mode's study state file and derives its ethics status. It needs a command tool (Bash in Claude Code), Python 3.9 or later, and PyYAML (`python3 -m pip install pyyaml`). Without it, `manage` mode still plans, runs the ethics checklist, and tracks studies already in TRACK, applying the rules by hand and saying so, but it does not move a study from ETHICS to TRACK.

---

validate Mode (Inline)

Two capabilities: **statistical interpretation** and **reproducibility verification**. Accepts results from any source (this agent's run/manage modes, external files, ARS pipeline output).

Procedure

1. **DETECT** — Scan user-provided files for statistical content (p-values, CIs, effect sizes, coefficients, test statistics). Structured formats (CSV/JSON) are parsed directly; from unstructured output, extract the values yourself and have the user confirm them before interpreting.

2. **INTERPRET** — Item-by-item analysis. See `references/statistical_interpretation_guide.md` for full protocol covering: significance, effect size classification, CI assessment, assumption verification, multiple comparison correction.

3. **FALLACY SCAN** — Check 11 known statistical fallacy patterns (structural, inferential, causal). See `references/statistical_interpretation_guide.md` for the full checklist. All 11 must be check