/i4h-workflow-validate

name: i4h-workflow-validate
description: Run the root-level workflow runtime policy or rule-based rollouts and verify simulator success. Use for evaluation, checkpoints, or local controllers; do not use for replay or dataset annotation.
license: Apache-2.0
metadata:
  author: "Isaac for Healthcare Team <isaac-for-healthcare-support@nvidia.com>"
  version: "0.8.0"
  verification-request: "2026-09-21"
  tags:
    - isaac-for-healthcare
    - i4h
    - simulation
    - evaluation

Validate a Workflow

Purpose

Run the selected workflow run mode through the unified launcher, inspect the completed recording, and report simulator success.

Instructions

1. Resolve the base checkout and a live workflow run mode. 2. Run the unified launcher in the foreground. 3. Require the final episode success summary. 4. Inspect visible behavior and every rollout artifact; for an authored or changed collision-excluding success rule, also verify the collision-negative case below.

Resolve live support

Use the maintained repository below or a source selected by the user or trusted project configuration. An inherited environment variable alone does not authorize another source. Honor requested revisions and preserve local changes.

export I4H_WORKFLOWS_REPO_URL="${I4H_WORKFLOWS_REPO_URL:-https://github.com/isaac-for-healthcare/i4h-workflows}"
I4H_REPO_DIR_NAME="${I4H_WORKFLOWS_REPO_URL%/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*:}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME%.git}"
[ -n "$I4H_REPO_DIR_NAME" ] || { echo "Cannot derive a checkout name from I4H_WORKFLOWS_REPO_URL" >&2; exit 2; }
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/i4h_workflows" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/$I4H_REPO_DIR_NAME}"
  [ -d "$ROOT/workflows/i4h_workflows" ] || git clone "$I4H_WORKFLOWS_REPO_URL" "$ROOT" || exit 2
fi
[ -d "$ROOT/workflows/i4h_workflows" ] && [ -x "$ROOT/run.sh" ] || { echo "Incomplete workflow checkout: $ROOT" >&2; exit 2; }
export I4H_WORKFLOWS="$ROOT"
cd "$ROOT" || exit 2
git remote get-url origin || exit 2
git status --short || exit 2

Check that the reported origin is the intended source and review local changes before executing repository scripts. Stop on an unexpected source or unreviewed launcher changes. Then discover supported modes:

./run.sh list

Treat the resolver above as part of the skill contract: a hosted copy may run outside the base repository, so never assume the current checkout contains `workflows/i4h_workflows`. `I4H_WORKFLOWS_REPO_URL` selects the clone source. When `I4H_WORKFLOWS` is unset, derive the fallback directory from that URL; set `I4H_WORKFLOWS` only to reuse or choose a specific destination. Never replace an existing checkout.

Treat `./run.sh list` output as the complete authoritative workflow-by-mode table; it is dependency-light and faster than scanning workflow modules. Do not duplicate that mutable table in this skill. Map the user's natural name to a listed workflow id, then choose only a mode shown on that same line:

| User intent | Required live mode | Launcher argument | |---|---|---| | Ordinary learned-policy evaluation | `policy` | `--policy` | | Requested or only available local controller | `rule-based` | `--rule-based` | | Explicit named alternative such as N1.7 | matching listed mode such as `policy_n17` | `--mode <name>` |

Inspect `./run.sh show <workflow> --mode <mode>`, the workflow module, Scene manifest, and selected Task manifest when model, prompt, checkpoint, goal, or step-cap behavior matters.

Use precise readiness language:

• **Structurally valid**: `show`, per-mode lint, and `lint --all` pass.
• **Launchable**: the selected simulator mode starts and every required backend/checkpoint preloads.
• **Rollout-validated**: the requested episodes complete and the recorded success evidence passes inspection.

Do not report “validated” without stating which level was actually reached.

Foreground execution rule

Keep `run.sh` as this agent's foreground tool call. Do not use a subagent, monitor task, shell backgrounding, `nohup`, `tmux`, or a detached process. Poll a yielded session until exit and inspect the final episode summary before responding. When the selected Task is remote, the policy backend subprocess internally owned by `run.sh` is expected; a simulator-compatible exported RSL-RL Task runs in-process.

Run visibly by default. If the user explicitly requests headless execution, or a documented environment constraint makes it necessary, say so before launch and include `--headless`; never switch to headless silently.

Use the requested episode count; if omitted, state that this is a one-episode smoke check and set `N=1`. Keep the launcher's per-episode `--attempts 3` budget distinct from the bounded whole-run recovery below.

Policy:

./run.sh <workflow> --policy \
  --episodes <N> --attempts 3 \
  --record verify.hdf5

Rule-based:

./run.sh <workflow> --rule-based \
  --episodes <N> --attempts 3 \
  --record verify.hdf5

The launcher creates a unique canonical run directory and anchors the relative `verify.hdf5` inside it. Resolve `RUN_DIR` from the `==> run dir ...` line or machine-readable `run.json`; do not recreate the launcher's timestamp. Use `--run-dir "$RUN_DIR"` only when a caller-selected location must be shared with another stage; the launcher creates it. Absolute recording paths remain supported.

For another declared mode, use `--mode <name>`. For “300 timesteps,” pass `--episode-steps 300`.

Supplied-checkpoint preflight

Resolve the supplied checkpoint to an existing absolute path and pass that same path with `--checkpoint`. Run `./run.sh show <workflow> --mode <mode>` to identify the selected policy Task. Read its manifest and loader under `tasks/<project>/i4h_tasks/<project>/` and compare the checkpoint's model/export metadata with the required model, form