/i4h-workflow-train-rl

name: i4h-workflow-train-rl
description: Use when training, evaluating, or exporting Workflow policies with online RSL-RL or RLinf, including RL checkpoint and Workflow handoff.
license: Apache-2.0
metadata:
  author: "Isaac for Healthcare Team <isaac-for-healthcare-support@nvidia.com>"
  version: "0.8.0"
  verification-request: "2026-09-21"
  tags:
    - isaac-for-healthcare
    - i4h
    - reinforcement-learning
    - rsl-rl
    - rlinf

Train a Workflow Policy with RL

Purpose

Resolve a maintained online-RL profile, verify its Scene/objective/model contracts, run the selected vectorized trainer, evaluate and export its artifact, and hand that artifact to normal Workflow policy validation.

Requirements

• Run the Workflow setup skill first so its uv environments and pinned third-party checkouts are available.
• Use a CUDA-capable Isaac Lab/Arena runtime for the RSL-RL workflow.
• For Trocar, provide a local GR00T N1.5 3B base or SFT checkpoint and two visible local GPUs for the isolated controller and simulator runtimes. A compatible checkpoint is a complete local Hugging Face directory that the pinned GR00T N1.5/RLinf loader accepts without conversion; it must retain the N1.5 3B architecture and support the maintained three-camera plus 28-joint observation mapping and 28-D policy action head. Reject another model family, an exported inference-only Task artifact, or a checkpoint whose config changes those interfaces.

Instructions

1. Resolve the checkout, confirm Workflow setup is complete, and select training, evaluation, or export from the request. Evaluation/export with an existing checkpoint skips training; never start a new training run to satisfy those requests. 2. For a maintained profile, proceed directly to profile and contract checks. Only a request to author a new RL-backed Workflow takes the authoring branch below; return to these checks after adding its profile. 3. Confirm the Workflow, Scene, observations, actions, rewards, resets, termination, trainer, and runtime Task contracts, then dry-run training or evaluation with its exact checkpoint and options. Export uses profile/contract checks and has no dry-run mode. 4. For training, preflight the selected trainer runtime and train in the foreground. For evaluation/export, go directly to the matching commands below using the supplied checkpoint or run bundle. 5. Evaluate before export and validate the export through the normal Workflow runner when those stages are in scope. Report any required validation that could not be completed.

Resolve the checkout

Before resolving the checkout, use the maintained repository below or an alternative already selected by the user or trusted project configuration. Check an existing checkout's origin and working-tree changes before executing its scripts; an inherited environment variable alone does not establish trust in an alternative source. Honor any requested revision and preserve local changes. If the source is unexpected, stop and resolve it before cloning or launching.

export I4H_WORKFLOWS_REPO_URL="${I4H_WORKFLOWS_REPO_URL:-https://github.com/isaac-for-healthcare/i4h-workflows}"
I4H_REPO_DIR_NAME="${I4H_WORKFLOWS_REPO_URL%/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*:}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME%.git}"
[ -n "$I4H_REPO_DIR_NAME" ] || { echo "Cannot derive a checkout name from I4H_WORKFLOWS_REPO_URL" >&2; exit 2; }
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/i4h_workflows" ]; then
  ROOT="${I4H_WORKFLOWS:-$HOME/$I4H_REPO_DIR_NAME}"
  [ -d "$ROOT/workflows/i4h_workflows" ] || git clone "$I4H_WORKFLOWS_REPO_URL" "$ROOT" || exit 2
fi
[ -d "$ROOT/workflows/i4h_workflows" ] && [ -x "$ROOT/run.sh" ] || { echo "Incomplete workflow checkout: $ROOT" >&2; exit 2; }
export I4H_WORKFLOWS="$ROOT"
cd "$ROOT" || exit 2

Treat this resolver as part of the skill contract. `I4H_WORKFLOWS_REPO_URL` selects the clone source; `I4H_WORKFLOWS` selects or reuses a checkout. Never replace an existing checkout.

Resolve the profile and contracts

./train.sh rl list
./train.sh rl show <workflow>
./run.sh show <workflow> --mode policy

Read the profile under `./rl/profiles/`, its referenced declarative trainer config under `./rl/config/`, Workflow, Scene manifest and implementation, Arena objective config, embodiment, and runtime policy Task manifest before a long run.

Select the maintained path from the profile:

| Workflow | Trainer | Starting artifact | Training observations/actions | Export and runtime | | --- | --- | --- | --- | --- | | `ultrasound_probe_reach` | RSL-RL PPO | None; train from scratch | 34-D joint/probe/target state → 6-D relative EE pose | TorchScript `policy.pt` → `rsl_rl/ultrasound_probe_reach` in-process Task | | `assemble_trocar` | RLinf PPO actor/critic | Local GR00T N1.5 SFT/base checkpoint | Three cameras + 28 arm/hand joints → 28 policy actions padded to the 43-D Scene action | Native RLinf run bundle → GR00T inference export → existing remote `gr00t_n15/assemble_trocar` Task |

For `ultrasound_probe_reach`, require the target to be sampled from verified upper-torso surface points, the table and phantom to remain fixed, success to require both position and orientation tolerance for consecutive steps, and the exported Task observation order to match training exactly.

For `assemble_trocar`, require Unitree G1 with Dex3 hands, `front` and both wrist cameras, the current 87-value body state (29 positions + 29 velocities + 29 torques), 14 Dex3 joint positions, the 28-D GR00T arm/hand mapping, the 15-value body-action prefix, and the maintained `g1_trocar` reward/termination contract. Do not copy another Trocar environment into the training tree.

Author a new RL-backed Workflow

1. Create and visibly validate the Workflow and Scene before adding training. Keep assets, embodiment, observations, actions, rewards, resets, t