/run-experiment

name: run-experiment
description: Runs existing Isaac Lab-Arena Experiment Definitions locally with experiment_runner.py in a ready native or Docker runtime, coordinates required local OpenPI or GR00T servers, applies local CLI overrides, and verifies generated result and report artifacts. Use for local named-Run or batch policy evaluations, variation listing, visualization or video, and result inspection. Do not use for installation or runtime preparation (setup-arena), pytest or regression checks (run-tests), interactive no-policy inspection (environment_runner.py), direct Policy Runner workflows, or OSMO preview, submission, or management.
allowed-tools: Read Grep Glob Skill Bash(git rev-parse --show-toplevel) Bash(id -un) Bash(test -d *) Bash(test -f *) Bash(test -x .venv/bin/python) Bash(env OMNI_KIT_ACCEPT_EULA=YES ACCEPT_EULA=Y .venv/bin/python isaaclab_arena/evaluation/experiment_runner.py *) Bash(docker ps *) Bash(docker exec *)

Run Experiment

Execute an existing Experiment Definition locally and finish with evidence from its canonical artifacts. Keep runtime readiness, regression testing, direct Policy Runner workflows, and managed submission outside this skill.

Read the checked-out documentation

Before composing a command, read:

• `docs/pages/concepts/concept_arena_experiments.rst` for Experiment structure and override rules.
• `docs/pages/quickstart/arena_experiment.rst` for the maintained local example.
• `docs/pages/quickstart/environment_variations.rst` when listing or applying variations.

Treat the current checkout as the source of truth. If this skill differs from the documentation or runner CLI, follow the checkout and report the mismatch.

Use typed YAML as the primary interface. Accept legacy JSON only as a pass-through compatibility path; do not create new legacy configurations or apply Hydra overrides to them.

Preflight the run

1. Confirm the repository root with `git rev-parse --show-toplevel`. 2. Resolve an explicit Experiment path. Never rely on the runner's legacy default configuration. 3. Inspect the Experiment's Runs, policies, rollout limits, rebuild counts, environment counts, cameras, variations, and referenced files. Summarize unexpectedly large work before starting it. 4. Preserve a native or Docker route already selected by the user. If both are ready and the request does not choose one, use the preferred route in the current installation documentation and state the choice. If neither route is ready, use `setup-arena`; do not install dependencies, build an image, create mounts, or recreate a container here. 5. Confirm that referenced configs, datasets, checkpoints, and output locations are available from the selected runtime. Identify a policy from each Run's resolved `policy.type`, not the Experiment filename. 6. For an OpenPI `Pi0RemotePolicy` or registered `pi0_remote` policy whose `remote_host` is local, use `serve-openpi-policy` for each distinct variant and port. It must reuse a compatible ready server or start one before this workflow launches the Experiment; the user does not need to invoke that skill separately. If Runs request different variants on the same port, stop and ask the user to resolve the conflict. 7. For a GR00T `Gr00tRemoteClosedloopPolicy` or registered `gr00t_remote_closedloop` policy whose `remote_host` is local, use `serve-gr00t-policy`. It may automatically start the maintained N1.6-DROID server or require an explicit model and embodiment for another client configuration; the Experiment does not declare the server checkpoint. If multiple Runs require incompatible server contracts on one port, resolve the conflict before launching. 8. For a supported policy pointing to another host, or another remote policy without a matching local server skill, require an already-running and reachable endpoint. Do not replace a deliberately remote endpoint with a local server or submit it to managed compute.

For a built-in smoke evaluation, use `isaaclab_arena_environments/experiment_configs/getting_started_experiment.yaml`. It uses the local zero-action policy and requires no model or policy server.

Select execution options

• Default to `--viz none` for unattended execution. Use `--viz kit` only when requested.
• Record viewport or camera video only when requested. Camera recording enables camera support and

can materially increase GPU memory and output size.

• Preserve the runner's default stop-on-first-error behavior. Add `--continue_on_error` only when

the user wants the remaining Runs attempted after a failure.

• Use `--serve_evaluation_report` only when explicitly requested; it binds an HTTP server and keeps

the process running until interrupted.

• Use `--list_variations` as an inspection operation. It does not run rollouts or create Experiment

result artifacts.

• Let the runner create a timestamped directory below `outputs/` unless the user chooses a base or

exact output location. An exact `--experiment_output_directory` must be missing or empty. Never clear or reuse a nonempty directory. If a requested exact directory is nonempty, stop and offer either a different exact directory or a fresh timestamped child through `--output_base_dir`; do not silently change the output semantics.

Build the local command

Always pass `--experiment_config` explicitly. Keep runner flags separate from trailing Hydra overrides.

For native uv, run from the repository root and accept the Isaac Sim EULA non-interactively:

env OMNI_KIT_ACCEPT_EULA=YES ACCEPT_EULA=Y \
  .venv/bin/python isaaclab_arena/evaluation/experiment_runner.py \
  --experiment_config <experiment.yaml> \
  --viz none \
  <override>...

For Docker, first use `docker ps` to select the single running container that mounts the absolute repository root. Resolve the host username separately with `id -un`, then substitute both literal values below rather than hardcoding a container name