/deepstream-profile-pipeline

name: "deepstream-profile-pipeline"
description: "Profile a DeepStream pipeline with Nsight Systems and derive its configs from the measurement. Use when the user asks for an efficient, performant, or profiled pipeline — or to benchmark, tune, or measure FPS."
metadata:
  author: "NVIDIA CORPORATION"
  tags:
    - deepstream
    - profiling
    - nsight-systems
    - nvtx
    - nvidia-smi
    - benchmarking
  languages:
    - bash
    - python
    - yaml
  domain: video-analytics
  team: deepstream-sdk
owner: "NVIDIA CORPORATION"
service: "deepstream"
version: "0.1.0"
reviewed: "2026-04-24"
license: CC-BY-4.0 AND Apache-2.0
compatibility: >
  DeepStream SDK 9.0 on Ubuntu 22.04 or 24.04, run from the
  `nvcr.io/nvidia/deepstream:9.0-triton-multiarch` container (the dev image; the slimmer
  `samples-multiarch` variant strips the nsys NVTX injector and produces empty per-plugin
  NVTX traces — do not use it for profiling). Requires `nsys` (Nsight Systems 2024+) and
  `nvidia-smi` on PATH. No GUI dependency — the skill runs fully headless and uses only
  `nsys profile` + `nsys stats`.
data_classification: "internal"

DeepStream Profiling Skill

Profile-driven pipeline creation. When the user indicates they want an efficient DeepStream pipeline, this skill replaces guesswork with two measured numbers — **inference plateau batch** and **HW ceiling** — and derives every other config from them. Then it profiles the E2E pipeline with Nsight Systems and reports per-plugin NVTX timings.

**Model- and pipeline-agnostic.** The skill assumes only that the inference element is `nvinfer` or `nvinferserver` (so model dims, precision, and batch knobs are settable through the standard config). It works for detection (with or without tracker), classification, segmentation, VLM, and embedding pipelines. Source can be file, RTSP, USB camera, or any mix. The skill reads the user's actual config to discover model dims / target FPS / source properties — it does NOT assume any particular model, codec, or resolution.

> **Constraint.** Terminal only. Use `nsys profile` to capture and `nsys stats` to extract. > Do not depend on Nsight Lens or any GUI.

When to trigger

Activate this skill **at pipeline creation time** when the user's ask carries efficiency intent. Concrete triggers:

• "build an **efficient** / **fast** / **performant** / **optimized** pipeline"
• "give me a pipeline that runs well on this GPU"
• "benchmark / profile / measure / tune / optimize this pipeline"
• "I want to run N streams at M FPS"
• "how many streams can this GPU handle"
• user explicitly asks for `nsys` or Nsight

For plain "build a pipeline" / "display this video" / "save this stream" with no perf intent, hand off to the `deepstream-generate-pipeline` skill instead.

The 6-stage flow

Run the stages in order. Stage 0 fires *before* the pipeline is generated, so the user starts from a perf-tuned skeleton. Stages 1–5 measure and verify.

Stage 0 — Preset-apply (at pipeline-creation time)

Trigger: any time the coding agent is about to generate a new DS pipeline AND the user's prompt carries efficiency intent (see "When to trigger" above).

Action: pre-apply these defaults *without prompting*. The user does not need to know any of them; they just get a pipeline that's already in the right shape.

| Knob | Default value | Skip when | |---|---|---| | `nvinfer.network-mode` | `1` (INT8) if a calibration file is present at `int8-calib-file=<path>`, else `2` (FP16). Never FP32. | Model has no INT8 calibration AND the user explicitly says "FP32". | | `nvinfer.model-engine-file` | Pre-built `.engine` path | Always set. Force a one-shot prebuild before measurement. | | `nvinfer.infer-dims` | `3;<H>;<W>` matching the model's native input | Always set, even for static-shape ONNX (harmless). | | `nvstreammux.batch-size` | `min(N_streams, 16)` until microbench refines it | — | | `nvstreammux.width / height` | model's native input dims (read from the nvinfer config's `infer-dims=3;H;W`) | User explicitly asks for native source resolution at the muxer. | | `nvstreammux.batched-push-timeout` | `1e6 / source_fps` µs (33333 for 30 fps) | — | | `nvstreammux.nvbuf-memory-type` | `0` (NVMM) | — | | Decoder `num-extra-surfaces` | `min(batch_size, 5)` | — | | Decoder `cudadec-memtype` | `0` (NVMM) | — | | Sink | `fakesink sync=False` for the benchmark variant | User asked for on-screen display or on-disk recording (then keep OSD/tiler/encoder/sink and produce TWO variants). | | OSD + tiler | omit | User asked for visible output. | | Tracker `ll-config-file` | `config_tracker_NvDCF_max_perf.yml` (perf-tuned NvDCF preset shipped with DS 9.0) | Tracker not present. | | Tracker `tracker-width / height` | 480 / 288 | — | | Tracker `enable-batch-process` (in linked YAML) | `1` | — | | Queue between source and pgie | `max-size-buffers = batch_size × 4` | No queue requested (rare). | | Kafka/message queue | `max-size-buffers=2, leaky=2` | No Kafka. | | Decode-side `PerfMonitor` | attach (in addition to pgie-side) | Pipeline is `nvurisrcbin → pgie` direct without intermediate queue. |

**Why Stage 0 exists:** without it, every newly generated pipeline starts from display-first defaults and Stages 1–5 spend cycles fixing avoidable issues. Stage 0 is the "don't write a bad pipeline in the first place" gate.

The student / API user never sees these knobs. The skill's response back to the user is in plain English (FPS, stream count, observed bottleneck), not knob names.

The verification flow (Stages 1–5)

Run the stages in order. Do not skip a stage — later stages depend on earlier ones' outputs.

Stage 1 — NVTX coverage check

DeepStream plugins emit NVTX ranges natively; custom plugins and plain GStreamer-core elements (`queue`, `tee`, `h264parse`, etc.) do not. Before profiling, list the elements the pipeline uses and classify each.

• Read the pipeline definition (gst-launch string or `pipeline.py`).
• For each element, look it up in [ref