/openreality

name: openreality
description: Drive Open Reality OS through the openreality MCP tools — scan upload, reconstruction, scene sync/query, scene agents, measurements, navigation, and robot-training exports. Use whenever the user mentions their Open Reality account, scans/scenes, splats, reconstructions, scene measurements, or LeRobot/GR00T/Isaac exports of captured spaces.

Open Reality OS

You have MCP tools (server name `openreality`) for a user's Open Reality account — persisted 3D scenes reconstructed from phone captures, imported splats, and robot recordings.

Workflow

1. **Sync first**: `workspace_list_scenes` — the account is the sync layer; scans made on the user's phone or the web appear here. `new_since_last_sync` tells you what's new. 2. **Context second**: `scene_card <scan_id>` — small, honest summary (source, metric state, objects, artifacts). @-mention `openreality://scene/<scan_id>` works too. 3. **Then act**:

• Deterministic questions (distances, angles, room structure, paths, exports) →

`scene_*` / `export_*` tools. These are pure server routes — no LLM cost.

• Narrated / visual questions ("describe this room", "key features") →

`agent_chat` / `agent_annotate`. These run the SERVER-side scene agent, which spends a bounded LLM budget from the account. Prefer `agent_replay` (always $0) when a recorded run already answers the question (`agent_runs` lists them). 4. **Uploads**: `workspace_upload_video` (≤1 GB) / `workspace_upload_recording` (robot .db) / `workspace_upload_splat` (.ply/.spz) → then `workspace_job_wait` on the job_id. 5. **Artifacts go to disk, never into context**: `artifact_fetch` / `artifact_fetch_splat` return local file paths. Exports: `export_manifest` (dry-run) → `export_prepare` → `artifact_fetch` of the returned key. Formats: `openreality`, `groot_lerobot_v2`, `isaac_usd`.

Honesty doctrine (non-negotiable — mirrors the product's own guardrails)

• **Units**: a length is metres ONLY when the response says `units: "m"` (a metric anchor

was applied — `scale_source` names it). `units: "relative"` means relative SLAM units: never call them metres, never convert. To unlock metres: `scene_anchor` with two picked points and their real distance.

• **Closed world**: the objects from `scene_list_objects` are the only objects that exist.

Don't invent, merge, or rename them beyond their labels.

• **Provenance rides along**: keyframes are photos; synthetic views are RENDERS of the

splat (say so); generated assets (SAM-3D completions, variants, floor patches) are generated — their meta.json says `generated: true`. Nav plans carry "not certified navigation" — repeat it when presenting a path.

• **Degraded is a state, not a failure**: robot-recording scenes persist degraded (no

detections, preview cloud) with a QC report under `derived/demo/recordings/` — report the QC verdict rather than hiding it.

• **The Isaac export is metric-gated**: without an anchor it is refused. That refusal is

correct behavior — offer the anchor flow, don't fight the gate.

• Surface server refusals as-is: 409 `agent_run_active` (one run per scene — attach or

wait), 422 `unreachable_goal` (includes the nearest reachable point), 409 `export_job_active`.

Troubleshooting

• `No Open Reality credentials` → run `openreality-mcp login --token <token>` (token from

the deployment's `/os` page URL hash after dashboard sign-in) or set `OPENREALITY_TOKEN`.

• Tokens roll automatically at half-life; a stale (>12 h idle) credential needs a fresh

login.

• To develop without the real backend: `openreality-mcp simulator`, then point the server

at it with `OPENREALITY_URL=http://127.0.0.1:8973 OPENREALITY_TOKEN=sim-token`.