/deploy-experiment

name: deploy-experiment
description: Deploy and run experiments for ML/DL training (local, remote, or Modal GPU) AND spatial data science / GIScience experiments (local, data-driven). Reads from output/refine-logs/EXPERIMENT_PLAN.md and output/refine-logs/FINAL_PROPOSAL.md, writes to output/experiment/. Use when user says "run experiment", "deploy experiment", "execute experiment plan", or needs to launch training / spatial analysis jobs.
argument-hint: [experiment-description-or-scope]
allowed-tools: Bash(*), Read, Grep, Glob, Edit, Write, Agent

Deploy Experiment

Execute the experiment described in **output/refine-logs/EXPERIMENT_PLAN.md** (with context from **output/refine-logs/FINAL_PROPOSAL.md**) and write all results to **output/experiment/**.

Scope override (optional): $ARGUMENTS

---

Inputs

| File | Purpose | |---|---| | `output/refine-logs/EXPERIMENT_PLAN.md` | Authoritative experiment plan: objectives, claims, run order, commands, success criteria, data needs | | `output/refine-logs/FINAL_PROPOSAL.md` | Final refined proposal: problem anchor, method, expected contribution, constraints | | `CLAUDE.md` | Environment config (gpu location, conda env, SSH, wandb, code_sync) |

If either required input is missing, stop and tell the user which file to produce first (likely via `/refine-research` and `/experiment-design`).

Outputs (all under `output/experiment/`)

| File | Content | |---|---| | `EXPERIMENT_RESULT.md` | Final results report: per-claim outcomes, tables, key numbers, figures referenced | | `EXPERIMENT_LOG.md` | Chronological run log: commands executed, environment, timings, stdout/stderr pointers, failures, retries | | `data/` | Intermediate data artifacts ready for visualization (CSV, Parquet, GeoJSON, GeoPackage, NetCDF, .npz, model predictions, metrics.json) | | `figures/` *(optional)* | Quick-look figures produced during the run (final paper figures come from `paper-figure`) | | `scripts/` *(optional)* | Any inline runner scripts created to execute the plan |

Create the `output/experiment/`, `output/experiment/data/`, `output/experiment/figures/`, and `output/experiment/scripts/` directories at the start if they do not exist.

---

Step 0: Mandatory Local GPU Availability Check (pilot AND full experiments)

**This check is REQUIRED before classification, regardless of whether the run is a pilot, a full experiment, Track A, Track B, or mixed.** The goal is to guarantee that when a local GPU is present, every ML / DL component runs on it — never silently on CPU.

Run the local GPU presence check first:

**Local CUDA:**

nvidia-smi --query-gpu=index,name,memory.used,memory.total --format=csv,noheader

**Local Mac MPS (only if `nvidia-smi` is missing or returns zero rows):**

python -c "import torch; print('MPS available:', torch.backends.mps.is_available())"

Decision rules (apply in order):

1. If `nvidia-smi` exits 0 and returns at least one GPU row → record `LOCAL_GPU=cuda`, capture the `<gpu_id>` and the chosen device for use in Step A5 and any pilot launch. Do **NOT** gate on idle memory; a desktop GPU with a display attached normally reports 0.7–1.5 GiB used. 2. Else if `torch.backends.mps.is_available()` returns `True` → record `LOCAL_GPU=mps`. 3. Else → record `LOCAL_GPU=none`. Only in this case may an ML / DL component fall back to CPU or to remote / Modal as defined in `CLAUDE.md`.

Write the result as the first lines of `output/experiment/EXPERIMENT_LOG.md`:

- LOCAL_GPU: cuda | mps | none
- Device chosen: <e.g. CUDA:0 / mps / cpu / remote-A100>
- GPU check command + output: <verbatim>

**Hard rule:** if `LOCAL_GPU` is `cuda` or `mps`, every ML / DL pilot AND every ML / DL full-experiment run launched by this skill MUST use that local GPU. Do not switch to remote / Modal / CPU just because the plan's default is set elsewhere — local GPU takes precedence whenever it is present. Record any deviation as a `CONTRACT_VIOLATION.md` with the user's explicit override.

If `LOCAL_GPU=none` and `CLAUDE.md` lists `gpu: remote` or `gpu: modal`, route ML / DL work there as documented in Step A1; otherwise stop and report to the user before proceeding.

This check applies equally to:

• Pilot experiments invoked from `/generate-idea` (Phase 5)
• Full experiments invoked from `/full-pipeline` Stage 2
• Any mixed GeoAI experiment whose Track A leg requires GPU

---

Step 0.1: Classify the Experiment

Read `EXPERIMENT_PLAN.md` and `FINAL_PROPOSAL.md`, then classify into ONE of the two tracks below. Record the choice at the top of `EXPERIMENT_LOG.md`.

| Track | Signals | Route | |---|---|---| | **A. ML / Deep Learning** | Model training, fine-tuning, gradient descent, epochs, GPU/TPU required, checkpoints, benchmarks on splits, wandb-style tracking | → Step A1 | | **B. Spatial / GIScience** | Spatial regression, ESDA (Moran's I, LISA, Getis-Ord), GWR/MGWR, spatial clustering, accessibility (2SFCA, isochrones), cartographic description, spatiotemporal analysis, network analysis, choropleth-driven claims, no GPU needed (mostly CPU + geospatial libs) | → Step B1 |

Mixed experiments (e.g., GeoAI: deep learning applied to spatial data) run **both** tracks in sequence — Track A for model training/inference, Track B for downstream spatial evaluation.

Invoke supporting skills when needed:

• If the plan requires datasets not yet in `data/raw/` → call **`data-download`** skill with the data requirements from the plan.
• For Track B (spatial analysis execution and diagnostics) → call **`spatial-analysis`** skill to run the analysis under guideline-driven decisions. This skill (deploy-experiment) orchestrates and records results; `spatial-analysis` is the executor for spatial methods.

---

Track A — ML / Deep Learning Experiments

A1. Detect Environment

From `CLAUDE.md`:

• **Local GPU** (`gpu: local`): local CUDA or Apple MPS
• **Remote server** (`gpu: remote`): SSH alias, conda env, code directory
• **Modal** (