/aris-run-experiment

name: aris-run-experiment
description: Deploy and run ML experiments on local, remote, Vast.ai, or Modal serverless GPU. Use when user says "run experiment", "deploy to server", "跑实验", or needs to launch training jobs.
argument-hint: "[experiment-description]"
allowed-tools: Bash(*), Read, Grep, Glob, Edit, Write, Agent, Skill(serverless-modal)
license: MIT
metadata:
  author: wanshuiyin/ARIS
  version: "1.0.0"

Run Experiment

Deploy and run ML experiment: $ARGUMENTS

Workflow

Step 0: Compute Resource Guard (MANDATORY)

**Before doing ANYTHING else**, run `/aris-compute-guard` to verify that compute resources are actually available.

If `/aris-compute-guard` is not available as a sub-skill, perform the check inline:

1. **Local GPU (Linux):** Run `nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader`. A GPU is free if `memory.used < 500 MiB`. 2. **Local GPU (Mac):** Run `python3 -c "import torch; print(torch.backends.mps.is_available())"`. 3. **Remote server:** Run `ssh <server> nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader`. 4. **Modal:** Check `modal token verify` — Modal is serverless and always available if configured.

**If compute resources are NOT available:**

• **STOP IMMEDIATELY.** Do NOT proceed to any further steps.
• Report to the user: which resources are missing, what they need to fix, and alternative options (Modal serverless, Vast.ai, remote server).
• **Do NOT fabricate, imagine, or hallucinate experiment results.** This is critical — running experiments without actual compute resources produces no real results.

**If compute resources ARE available:** Print a brief confirmation and proceed to Step 1.

Step 1: Detect Environment

Read the project's `CLAUDE.md` to determine the experiment environment:

• **Local GPU** (`gpu: local`): Look for local CUDA/MPS setup info
• **Remote server** (`gpu: remote`): Look for SSH alias, conda env, code directory
• **Vast.ai** (`gpu: vast`): Check for `vast-instances.json` at project root — if a running instance exists, use it. Also check `CLAUDE.md` for a `## Vast.ai` section.
• **Modal** (`gpu: modal`): Serverless GPU via Modal. No SSH, no Docker, auto scale-to-zero. Delegate to `/aris-serverless-modal`.

**Modal detection:** If `CLAUDE.md` has `gpu: modal` or a `## Modal` section, the entire deployment is handled by `/aris-serverless-modal`. Jump to **Step 4: Deploy (Modal)** — Steps 2-3 are not needed (Modal handles code sync and GPU allocation automatically).

**Vast.ai detection priority:** 1. If `CLAUDE.md` has `gpu: vast` or a `## Vast.ai` section:

• If `vast-instances.json` exists and has a running instance → use that instance
• If no running instance → call `/aris-vast-gpu provision` which analyzes the task, presents cost-optimized GPU options, and rents the user's choice

2. If no server info is found in `CLAUDE.md`, ask the user.

Step 2: Pre-flight Check

Check GPU availability on the target machine:

**Remote (SSH):**

ssh <server> nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader

**Remote (Vast.ai):**

ssh -p <PORT> root@<HOST> nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader

(Read `ssh_host` and `ssh_port` from `vast-instances.json`, or run `vastai ssh-url <INSTANCE_ID>` which returns `ssh://root@HOST:PORT`)

**Local:**

nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader
# or for Mac MPS:
python -c "import torch; print('MPS available:', torch.backends.mps.is_available())"

Free GPU = memory.used < 500 MiB.

Step 3: Sync Code (Remote Only)

Check the project's `CLAUDE.md` for a `code_sync` setting. If not specified, default to `rsync`.

Option A: rsync (default)

Only sync necessary files — NOT data, checkpoints, or large files:

rsync -avz --include='*.py' --exclude='*' <local_src>/ <server>:<remote_dst>/

Option B: git (when `code_sync: git` is set in CLAUDE.md)

Push local changes to remote repo, then pull on the server:

# 1. Push from local
git add -A && git commit -m "sync: experiment deployment" && git push

# 2. Pull on server
ssh <server> "cd <remote_dst> && git pull"

Benefits: version-tracked, multi-server sync with one push, no rsync include/exclude rules needed.

Option C: Vast.ai instance

Sync code to the vast.ai instance (always rsync, code dir is `/workspace/project/`):

rsync -avz -e "ssh -p <PORT>" \
  --include='*.py' --include='*.yaml' --include='*.yml' --include='*.json' \
  --include='*.txt' --include='*.sh' --include='*/' \
  --exclude='*.pt' --exclude='*.pth' --exclude='*.ckpt' \
  --exclude='__pycache__' --exclude='.git' --exclude='data/' \
  --exclude='wandb/' --exclude='outputs/' \
  ./ root@<HOST>:/workspace/project/

If `requirements.txt` exists, install dependencies:

scp -P <PORT> requirements.txt root@<HOST>:/workspace/
ssh -p <PORT> root@<HOST> "pip install -q -r /workspace/requirements.txt"

Step 3.5: W&B Integration (when `wandb: true` in CLAUDE.md)

**Skip this step entirely if `wandb` is not set or is `false` in CLAUDE.md.**

Before deploying, ensure the experiment scripts have W&B logging:

1. **Check if wandb is already in the script** — look for `import wandb` or `wandb.init`. If present, skip to Step 4.

2. **If not present, add W&B logging** to the training script:

   import wandb
   wandb.init(project=WANDB_PROJECT, name=EXP_NAME, config={...hyperparams...})

   # Inside training loop:
   wandb.log({"train/loss": loss, "train/lr": lr, "step": step})

   # After eval:
   wandb.log({"eval/loss": eval_loss, "eval/ppl": ppl, "eval/accuracy": acc})

   # At end:
   wandb.finish()

3. **Metrics to log** (add whichever apply to the experiment):

• `train/loss` — training loss per step
• `train/lr` — learning rate
• `eval/loss`, `eval/ppl`, `eval/accuracy` — eval metrics per epoch
• `gpu/memory_u