/deepstream-import-vision-model

name: deepstream-import-vision-model
description: >
  Use this skill to bring a supported object-detection vision model from HuggingFace or NVIDIA NGC into
  an NVIDIA DeepStream pipeline with end-to-end automation: ONNX download,
  SafeTensors export, TRT engine build, custom nvinfer bbox parser, multi-stream
  benchmark, and PDF report. Object detection models only.
license: CC-BY-4.0 AND Apache-2.0
metadata:
  author: "Tushar Khinvasara <tkhinvasara@nvidia.com>"
  owner: "Tushar Khinvasara <tkhinvasara@nvidia.com>"
  service: "deepstream"
  version: "1.5.2"
  reviewed: "2026-08-04"
  team: deepstream-sdk
  tags:
    - deepstream
    - tensorrt
    - object-detection
    - import-vision-model
  languages:
    - bash
    - python
    - cpp
  domain: computer-vision

DeepStream Import Vision Model

When this skill is active, **read the relevant reference document before starting each phase**. Do not rely on memory — reference documents contain exact script paths, bash variable conventions, log filename contracts, and critical parsing rules.

**Current scope:** Object detection models only. Fail fast on classification, segmentation, or other architectures detected in `config.json`.

Model choice — always offer two options

Before preflight, browsing, downloads, or file creation, present exactly these two choices. Do not start with only an open-ended model-source prompt. If the user's request already clearly selects a model, confirm the matching choice instead of asking redundantly.

1. Default model (recommended)

Use the validated Hugging Face RT-DETR model:

model_id: PekingU/rtdetr_r50vd
source: huggingface
task: object-detection
precision_preference: fp16

2. Custom object-detection model

Ask for one supported source:

• Hugging Face model ID (`organization/model`) or full model URL.
• NVIDIA NGC catalog model URL including its version.

Explain that the skill currently rejects classification, segmentation, and other non-detection architectures after inspecting `config.json`. Do not invent or silently substitute a model when the custom source is missing or unsupported.

For a dry run, present the same two choices and simulate discovery, build, benchmark, and report stages without browsing, downloading, launching Docker, writing files, or starting processes.

Pipeline Overview

| Step | Phase | Reference | What it does | |------|-------|-----------|--------------| | 1–3 | Model Acquire | [references/model-acquire.md](references/model-acquire.md) | Browse HF/NGC, detect format, download ONNX or export SafeTensors | | 4–5 | Engine Build | [references/engine-build.md](references/engine-build.md) | Build dynamic TRT engine, run trtexec BS=1 and BS=MAX_BS | | 6–7 | DS Pipeline | [references/pipeline-run.md](references/pipeline-run.md) | Custom bbox parser, nvinfer config, single-stream + multi-stream benchmarks | | 8 | Report | [references/report-generation.md](references/report-generation.md) | 5 charts, HTML, PDF benchmark report |

Run the full pipeline autonomously without pausing for confirmation at each step.

Runs entirely through Docker (no host packages)

**Every step runs INSIDE the DeepStream container.** The host needs only **Docker + the NVIDIA driver** — no host python/venv/torch/trtexec/make/wkhtmltopdf. This works identically on Linux and **Windows** (Docker Desktop + WSL2 backend, required for `--gpus`). The per-shell bind-mount token is the only OS difference — `-v "$PWD":/work` (bash), `-v "${PWD}:/work"` (PowerShell), `-v "%cd%:/work"` (cmd); full guide in [references/windows.md](references/windows.md). All venv/ONNX/ engine/parser/config/report artifacts live under the mounted working root and persist between the ephemeral `--rm` containers.

Pre-flight — bootstrap + verify (through the container)

**1. One-time bootstrap** — builds `build/.venv_optimum` (torch/onnx/onnxruntime/report deps; the venv name is historical, optimum is no longer used) + installs `wkhtmltopdf`, all in-container. From the working root:

docker run --rm -it --gpus all --shm-size=16g -v "$PWD":/work -w /work \
  --entrypoint bash nvcr.io/nvidia/deepstream:9.1-triton-multiarch \
  .claude/skills/deepstream-import-vision-model/setup.sh

**2. Preflight** — GPU + venv + trtexec, run THROUGH the container (container-mode auto-detects):

docker run --rm --gpus all -v "$PWD":/work -w /work \
  --entrypoint bash nvcr.io/nvidia/deepstream:9.1-triton-multiarch \
  .claude/skills/deepstream-import-vision-model/scripts/preflight.sh   # proceed only on PASS

**Every subsequent phase runs the same way** — issue the model's commands via `docker run … --entrypoint bash … -lc '<commands>'` (or the `.claude/skills/deepstream-import-vision-model/scripts/dsrun.sh` wrapper: `bash .claude/skills/deepstream-import-vision-model/scripts/dsrun.sh '<in-container command>'`), using `PY=build/.venv_optimum/bin/python` and `trtexec` at `/usr/src/tensorrt/bin/trtexec` inside the container. `deepstream-app`, `gst-launch-1.0`, and `/opt/nvidia/deepstream/…` sample paths all exist **in** the image. TensorRT build+runtime share one image, so there is **no version skew** (the concern the old "build on the host" rule tried to avoid — see [references/engine-build.md](references/engine-build.md)). `sample_720p.mp4` ships in the image; set `DS_VIDEO` only to override.

Mandatory Output Structure

Create once `MODEL_NAME` is known (Step 1). Never dump files flat.

models/{model_name}/
  model/           <- ONNX file(s)
  parser/          <- .cpp, Makefile, .so
  config/          <- nvinfer config, ds-app config, labels.txt
  scripts/         <- run helper scripts
  benchmarks/
    engines/       <- _dynamic_b{MAX_BS}.engine, timing.cache, build logs
    b1/            <- trtexec BS=1 log
    b{MAX_BS}/     <- trtexec BS=MAX_BS log
    ds/            <- DS benchmark logs
  reports/         <- benchmark_report.md, .html, .pdf, benchmark_data.json
    chart