nemo-relay

NVIDIA NeMo Relay

NVIDIA NeMo Relay provides visibility into and control over agent runs without requiring changes to the existing agent stack. It gives coding agents, applications, framework integrations, middleware, and observability backends a shared runtime for scopes, policy, plugins, and lifecycle events.

For how Relay complements OpenTelemetry GenAI conventions and observability or evaluation products, see the Ecosystem guide.

Where To Start

Goal	Start With
Observe Codex or Claude Code locally with the CLI	Quick Start CLI
Instrument app-owned LLM or tool calls	Quick Start Application
Use LangChain, LangGraph, Deep Agents, or OpenClaw	Supported Integrations
Build a framework or provider integration	Integrate into Frameworks
Export ATOF events, ATIF trajectories, or OpenTelemetry traces, logs, and metrics	Observability Plugin
Package reusable middleware or exporters	Build Plugins
Develop or test this repository from source	CONTRIBUTING.md

Hermes Agent understands NeMo Relay plugin configurations. NeMo Relay is built into Hermes Agent, so no separate observability plugin or Relay CLI setup is required.

Quick Start CLI

Start by recording a real agent run on disk. After Relay writes raw events and a trajectory file, you have concrete data to inspect, debug, and build on.

Local Agent Trajectory

This walkthrough shows an end-to-end quick success setup. Install the NeMo Relay CLI, turn on local exporters, run Codex or Claude Code through Relay, and check that Relay wrote both raw events and normalized trajectories.

1. Install the CLI

Install the prebuilt CLI from PyPI:

pip install nemo-relay-cli-bin

Python API users can install the matching CLI through the optional extra:

pip install "nemo-relay[cli]"

Alternatively, run the installer for your platform:

curl -fsSL https://raw.githubusercontent.com/NVIDIA/NeMo-Relay/main/install.sh | sh

irm https://raw.githubusercontent.com/NVIDIA/NeMo-Relay/main/install.ps1 | iex

Verify that the installed binary is available:

nemo-relay --version

The installer supports Linux x86_64/ARM64, macOS Apple Silicon, and Windows x86_64/ARM64. Refer to the installation guide for version pinning, custom directories, uninstalling a script-installed CLI, and source-based installation.

2. Enable Local Observability Output

Open the user-scoped plugin editor:

nemo-relay plugins edit

The editor creates or updates $XDG_CONFIG_HOME/nemo-relay/plugins.toml (or ~/.config/nemo-relay/plugins.toml). In the top-level menu, select Observability, then configure these sections:

1. Toggle the Observability component on.

2. Open ATOF. Toggle the section [on].

   Add a file sink. You can set its output_directory to .nemo-relay/atof, filename to events.jsonl, and mode to overwrite. Add stream sinks to send the same events to remote collectors.

3. Open ATIF. Toggle the section [on].

   Optionally set:

   • output_directory to .nemo-relay/atif
   • filename_template to trajectory-{session_id}.json

4. Return to the top-level menu and press p to preview the generated TOML.

5. Press s to save.

[!NOTE] Repository-local .nemo-relay/plugins.toml files are ignored. To use a configuration stored elsewhere, pass --config path/to/config.toml; Relay also selects the sibling path/to/plugins.toml.

3. Run a Coding Agent Through Relay

Run the Relay wrapper for the host CLI installed on your machine. For example:

nemo-relay codex -- exec "Summarize this repository."

For Claude Code, run:

nemo-relay claude -- "Summarize this repository."

Refer to the full Quick Start CLI docs for more options.

The transparent wrapper starts a local Relay gateway, injects host-specific hook and provider settings for that launched process, then shuts the gateway down when the agent exits.

[!WARNING] The transparent wrapper trusts only the Codex hooks that it generates for the launched process. A persistent nemo-relay install codex trusts only the hooks owned by nemo-relay-plugin@nemo-relay-local; manual and source marketplace installs can still require review. Restart an open Codex app after persistent installation.

On Windows, a restrictive host Job Object can keep the shared Relay gateway within the host process lifetime. If the host also rejects the required nested assignment, persistent bootstrap stops and explains the conflict. The Codex desktop app has additional limitations. Refer to the Codex CLI guide for lifecycle, startup, and troubleshooting details.

4. Verify the Run

After the run exits, check that raw events and trajectory files were written. If the optionally set output directory and file name were used:

test -s .nemo-relay/atof/events.jsonl
ls .nemo-relay/atif/*.json
for file in .nemo-relay/atif/*.json; do
  python3 -m json.tool "$file" >/dev/null
done

Then verify that at least one raw ATOF 0.1 event exists:

python3 - <<'PY'
from pathlib import Path
import json

events_path = Path(".nemo-relay/atof/events.jsonl")
events = [
    json.loads(line)
    for line in events_path.read_text().splitlines()
    if line.strip()
]

assert events, "no ATOF events were written"
assert any(event.get("atof_version") == "0.1" for event in events), "no ATOF 0.1 events found"
print(f"validated {len(events)} ATOF event(s)")
PY

A successful run creates several outputs to inspect:

• .nemo-relay/atof/events.jsonl as the raw canonical event stream.
• One or more .nemo-relay/atif/*.json trajectory files for analysis and evaluation workflows.

[!TIP] If raw ATOF events exist but LLM spans are missing, provider traffic probably isn't flowing through the Relay gateway. If ATIF is missing, make sure the agent session or turn ended and the output directory is writable.

Next Steps

Refer to the full NeMo Relay CLI docs for persistent host plugin installation, gateway configuration, exporter options, and agent-specific diagnostics.

[!TIP] Start by trusting the raw Agent Trajectory Observability Format (ATOF) JSONL. It shows the lifecycle events Relay actually captured before anything is translated into Agent Trajectory Interchange Format (ATIF) or OpenTelemetry traces, logs, and metrics, including the OpenInference projection.

Quick Start Applications

If writing the code that calls the model or tool, install the binding for the appropriate language and route that boundary through Relay directly.

Application Trajectory

Install Relay for the application language:

# Python
uv add nemo-relay

# Node.js
# Requires Node.js 24 or newer.
npm install nemo-relay-node@0.11.0

# Rust
cargo add nemo-relay

Then run a minimal example workflow for that binding:

• Python Quick Start
• Node.js Quick Start
• Rust Quick Start

What Relay Adds

Relay connects agent systems. A production application can combine NeMo Agent Toolkit, LangChain, LangGraph, provider SDKs, custom harness code, NeMo Guardrails, tracing systems, and evaluation pipelines. Relay gives those pieces one runtime contract instead of asking every layer to invent its own wrappers and trace vocabulary.

Relay gives those systems:

• Scopes so runs, turns, tools, LLM calls, and subagents have clear ownership, parent-child lineage, cleanup boundaries, and request isolation.
• Managed LLM and tool calls so the same lifecycle and middleware rules apply around each callback.
• Middleware for the places where Relay must block, sanitize, transform, route, retry, or replace execution.
• Plugins so reusable observability, guardrail, adaptive, and exporter behavior can be turned on from configuration.
• Events and subscribers so raw ATOF, normalized ATIF, and OpenTelemetry traces, logs, and metrics all come from the same runtime stream.

Relay does not replace frameworks, model provider, application logic, observability backend, or guardrail authoring system. It gives those systems a common boundary to meet.

flowchart LR
    App[Application, Framework, or CLI Harness]

    subgraph Runtime[NeMo Relay Runtime]
        direction TB
        Scopes[Scopes]
        Middleware[Middleware]
        Plugins[Plugins]
        Events[Lifecycle Events]
    end

    Output[Subscribers and Exporters]

    App --> Scopes
    App --> Middleware
    Plugins --> Middleware
    Scopes --> Events
    Middleware --> Events
    Events --> Output

Support Status

[!NOTE] The main supported paths today are Rust, Python, and Node.js. Go and raw C FFI are available for source-first users, but they are still experimental.

The following table shows which language bindings and CLI features are currently supported:

Binding	Status	Notes
Python	Fully supported	Documented with Quick Start and Guides.
Node.js	Fully supported	Documented with Quick Start and Guides.
Rust	Fully supported	Documented with Quick Start and Guides.
NeMo Relay CLI	Supported	Local observability and hook-backed security are supported; optimization is partial and host-dependent.
Go	Experimental	Source-first under go/nemo_relay.
FFI	Experimental	Source-first under crates/ffi.

Agent Harness Support

The CLI support matrix separates the supported CLI surface from host-specific coverage.

• Observability works for the listed harnesses.