/ambient-healthcare-agent-with-nemotron-voice-agent

name: ambient-healthcare-agent-with-nemotron-voice-agent
description: Customize NVIDIA Nemotron Voice Agent's Generic Pipecat example for healthcare appointment, five-field patient intake, or custom tool-calling workflows without a separate backend.
license: CC-BY-4.0 AND Apache-2.0
allowed-tools: Read Grep Glob Edit Write Bash Env WebFetch
metadata:
  author: "Jin Li <jinl@nvidia.com>"
  team: healthcare-tme
  tags: ambient-healthcare,nemotron-voice-agent,pipecat,tool-calling,healthcare
  version: "1.0.0"

Ambient Healthcare Agent with Nemotron Voice Agent

Purpose

Customize `src/examples/generic/` in a user-selected NVIDIA Nemotron Voice Agent (NVA) checkout. Use this skill for appointment making, five-field patient intake, or a developer-defined healthcare workflow that fits one Pipecat pipeline with an LLM prompt, OpenAI-style tool schemas, and Python handlers.

This skill is owned by Healthcare TME. It is not maintained or endorsed by the NVA team, and it does not live in the NVA repository. For ordinary NVA deployment or non-healthcare voice work, follow the public NVA documentation instead.

Read [the user-experience flowchart](references/user-experience-flowchart.md) when a visual overview of the gated workflow or bundled scenario state machines would help.

Requirements

• A compatible checkout of `https://github.com/NVIDIA-AI-Blueprints/nemotron-voice-agent`
• Python 3.10+ with PyYAML, Git, and Docker Compose
• Network access and credentials for the user-selected LLM, ASR, and TTS services
• Permission to write only to the NVA checkout path the user explicitly supplies
• Human review before sending fictional healthcare-style histories to any model endpoint

Treat the skill loader's installed directory as `SKILL_DIR`. Resolve bundled `scripts/` and `references/` from that directory, not from the current working directory.

Required Welcome

Begin a new positive workflow with this phrase verbatim:

Welcome to the NVIDIA Nemotron Voice Agent (NVA for short). We will customize NVA for creating ambient healthcare agents.
Now I will make a fresh clone of the Nemotron Voice Agent repository, and this will be the directory we work out of. Where would you like me to clone the repo to? Please provide a path.
If you already have a clone of the repository somewhere, please point me to the path.

Then stop. Do not inspect files, search for clones, reuse a path from earlier context, run tools, or choose a default. Set `NVA_ROOT` only from a path the user provides or confirms after this welcome.

Instructions

1. Resolve and validate the NVA checkout

If the user requests a fresh clone, clone only to their exact destination and only with network and write permission:

git clone https://github.com/NVIDIA-AI-Blueprints/nemotron-voice-agent.git "$NVA_ROOT"

If cloning fails, report the observed reason and ask the user to either grant the session the required access via `/permissions` and request a retry, or manually clone the NVA repository and provide its path. Do not retry until the user grants access or supplies a checkout path. For either a fresh or existing checkout, require these markers:

test -f "$NVA_ROOT/docker-compose.yml" && \
test -f "$NVA_ROOT/examples_registry.yaml" && \
test -d "$NVA_ROOT/src/examples/generic"

An invalid path is a hard stop. Do not modify any repository before this check passes.

After the markers pass, create `$NVA_ROOT/.env` by copying `$NVA_ROOT/.env.example` when the target does not already exist. Preserve an existing `.env` and never print its contents. If `.env` is absent and the template is missing, report that setup failure and stop. Do this immediately after validating a fresh clone or an existing checkout, before asking the user to choose hosted services.

2. Inspect compatibility and service defaults

Run:

python3 "$SKILL_DIR/scripts/inspect_nva_generic_defaults.py" \
  --nva-root "$NVA_ROOT" --check-compatibility

Stop on compatibility errors. Report the inspected prompt plus the LLM, ASR, and TTS key, display name, model/server, base URL when present, and catalog source. Never substitute release-specific defaults from memory.

The preflight checks Python structure and required capabilities rather than an NVA version number or one exact source string. It also discovers deployment skills from both `skills/*/SKILL.md` and `.agents/skills/*/SKILL.md`. A passing check is not permission to guess through an unknown layout: stop when syntax or a required semantic hook is ambiguous.

3. Select the runtime and verify setup access

Explain that the default is the compatible public NVIDIA AI Endpoint entries in the NVA cloud catalog. Before any inference, let the user choose public NVIDIA endpoints, NVA-managed local NIMs, existing NIM endpoints, or a mixed layout. Never silently fall back to public endpoints after opt-out. In the same message as these choices, give the user the actual absolute path to `$NVA_ROOT/.env` and tell them to fill in its `NVIDIA_API_KEY=` entry if they choose public NVIDIA AI Endpoints. Explain that the key authenticates access to those endpoints. Direct the user to the public NVA deployment documentation for credential setup; never ask them to paste a secret into chat or display the file contents. State:

The NVIDIA_API_KEY is required to utilize public NVIDIA AI Endpoints. With this key configured, I will be running live tests while customizing and standing up a Nemotron Voice Agent application.

Before applying a healthcare overlay:

1. Run `python3 "$SKILL_DIR/scripts/verify_docker_compose_access.py"` and require all checks to pass. If Docker Compose access is blocked by session permissions, report the observed failure, ask the user to grant the required access via `/permissions`, and stop until the user requests a retry. 2. Use the public NVA deployment instructions to identify one recipe and the selected runtime's credential and endpo