/deepstream-run-mv3dt

name: "deepstream-run-mv3dt"
description: "Run and operate the DeepStream Multi-View 3D Tracking reference app, also known as MV3DT. Use when the user asks to set up prerequisites, run shipped MV3DT samples, run Multi-View 3D Tracking on custom synchronized MP4 datasets, import camera calibration, delegate missing calibration to AutoMagicCalib, inspect OSD or BEV visualization, consume MV3DT Kafka metadata, or clean up MV3DT run state in the DeepStream MV3DT app directory."
metadata:
  author: "Shubham Agrawal <shuagrawal@nvidia.com>"
  tags: [deepstream, mv3dt, tracking, multi-view, 3d, kafka]
  languages: [bash, python]
  domain: computer-vision
owner: "NVIDIA CORPORATION"
service: "deepstream-tracker-3d-multi-view"
version: "1.0.0"
reviewed: "2026-06-15"
license: CC-BY-4.0 AND Apache-2.0

Skill: Run DeepStream MV3DT

When to Use This Skill

Activate this skill when the user wants to set up, run, verify, or debug the DeepStream Multi-View 3D Tracking reference app. Typical prompts:

• "set up MV3DT DeepStream"
• "run the 4-camera MV3DT sample"
• "run the 12-camera MV3DT sample"
• "run MV3DT on my synchronized MP4s"
• "I have videos but no calibration; calibrate and run MV3DT"
• "show the BEV visualizer or Kafka metadata"
• "stop MV3DT" / "clean up MV3DT" / "tear down MV3DT"

Do not use this skill for single-view 3D tracking, generic DeepStream app development, or live-stream onboarding unless the user explicitly maps that work to this repo's MV3DT pipeline.

Examples

• "Deploy the DeepStream MV3DT 4-camera sample and show the OSD and BEV windows."
• "Run the MV3DT 12-camera sample headlessly with RTDETR and save videos."
• "Run MV3DT on synchronized MP4s under /data/mv3dt-demo using PeopleNetTransformer."
• "My custom MV3DT videos do not have calibration; use AutoMagicCalib, then run MV3DT."

Overview

Operate the Multi-View 3D Tracking reference app in `DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view` using the DeepStream Container path. The skill supports setup, shipped sample runs, custom synchronized MP4 datasets, calibration handoff to AutoMagicCalib, display/headless execution, OSD/BEV outputs, and Kafka metadata inspection.

Prerequisites

• MV3DT reference app directory on disk under `DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view`; if it is absent, ask before cloning the public DeepStream repo
• Docker with NVIDIA GPU support
• DeepStream container image access
• MV3DT sample datasets, models, custom parser libraries, Kafka, Mosquitto, and `mv3dt_venv` prepared by the repo setup script
• Working X11/VNC display for live OSD/BEV windows, or the saved-output headless path for tiled DeepStream MP4 plus Kafka-derived BEV MP4

Instructions

Step 0: Resolve MV3DT App Checkout

The skill can be installed outside the DeepStream repo. Resolve `REPO_ROOT` to the MV3DT app directory, not necessarily the Git top-level directory.

MV3DT_APP_SUBDIR="src/apps/reference_apps/deepstream-tracker-3d-multi-view"

is_mv3dt_app_dir() {
  test -f "$1/README.md" || return 1
  test -d "$1/config_templates" || return 1
  test -d "$1/scripts" || return 1
  grep -q "Multi-View 3D Tracking" "$1/README.md"
}

GIT_TOP="$(git rev-parse --show-toplevel 2>/dev/null || true)"
CANDIDATES=()
if [ -n "${MV3DT_REPO_ROOT:-}" ]; then CANDIDATES+=("${MV3DT_REPO_ROOT}"); fi
if [ -n "${DEEPSTREAM_REPO_ROOT:-}" ]; then CANDIDATES+=("${DEEPSTREAM_REPO_ROOT}/${MV3DT_APP_SUBDIR}"); fi
CANDIDATES+=("${PWD}")
if [ -n "${GIT_TOP}" ]; then
  CANDIDATES+=("${GIT_TOP}" "${GIT_TOP}/${MV3DT_APP_SUBDIR}")
fi
CANDIDATES+=("${HOME}/DeepStream/${MV3DT_APP_SUBDIR}" "${HOME}/deepstream/${MV3DT_APP_SUBDIR}")

REPO_ROOT=""
for candidate in "${CANDIDATES[@]}"; do
  if [ -n "$candidate" ] && is_mv3dt_app_dir "$candidate"; then
    REPO_ROOT="$(cd "$candidate" && pwd)"
    break
  fi
done

if [ -z "${REPO_ROOT}" ]; then
  cat <<'EOF'
ERROR: MV3DT reference app directory was not found.
Set MV3DT_REPO_ROOT to an existing deepstream-tracker-3d-multi-view app directory, or ask the user to approve cloning the public DeepStream repo and then run:

  DEEPSTREAM_REPO_ROOT="${DEEPSTREAM_REPO_ROOT:-$HOME/DeepStream}"
  git clone https://github.com/NVIDIA/DeepStream.git "$DEEPSTREAM_REPO_ROOT"
  export MV3DT_REPO_ROOT="$DEEPSTREAM_REPO_ROOT/src/apps/reference_apps/deepstream-tracker-3d-multi-view"

Do not clone silently.
EOF
  exit 1
fi

cd "${REPO_ROOT}"
export REPO_ROOT MV3DT_REPO_ROOT="${REPO_ROOT}"

If the app directory cannot be resolved, ask the user for an existing checkout path or for approval to clone `https://github.com/NVIDIA/DeepStream`. Do not clone silently.

Step 1: Select The Primary Workflow

Load exactly one primary reference for the user's current request:

| User intent | Reference | |---|---| | Install, prepare, or verify prerequisites | `references/setup.md` | | Run bundled 4-camera or 12-camera sample | `references/sample-run.md` | | Run custom synchronized MP4s | `references/custom-dataset.md` | | Missing custom calibration | `references/amc-calibration-handoff.md`, then return to `references/custom-dataset.md` | | View OSD, BEV, screenshots, recordings, or Kafka metadata | `references/visualization-metadata.md` | | Stop a run, clean generated artifacts, or stop prerequisite services | `references/setup.md` |

If setup, datasets, models, Kafka, Mosquitto, Docker GPU runtime, or the Python venv are missing, load `references/setup.md` before continuing to the user's original workflow.

When `sample-run.md` or `custom-dataset.md` needs to regenerate DeepStream configs, load `references/generate-configs.md` as the canonical shared config-generation reference rather than duplicating the shell logic.

Step 2: Follow The Run Stages

For every run, use this stage order:

| Stage | Action | |---|---| | Validate | Check app directory, prerequisites, dataset shape, display/headless mode, Docker GPU support, and output-directory writability. | | Prepar