/doca-gpunetio-ib-write-bw

license: Apache-2.0
name: doca-gpunetio-ib-write-bw
description: >
  Use this skill when the user is building, running, or interpreting
  the doca/tools/gpunetio_ib_write_bw client+server benchmark — a CUDA
  kernel on the client posts RDMA WRITE work requests through the
  doca-gpunetio device-side surface to measure sustained GPU-driven
  WRITE bandwidth on a GPU+IB-device pair. Trigger even when the user
  does not explicitly mention "doca-gpunetio-ib-write-bw" or
  "GPUNetIO" — typical implicit phrasings include "measure WRITE BW
  when the GPU posts the WRs", "BW swings between runs on the same
  flags", "is the NIC saturated or am I CPU-bound on the CUDA
  kernel", "meson compile fails for the GPUNetIO bw tool",
  "nvidia_peermem isn't picking up my GPU buffer", or "GPU-initiated
  WRITE throughput vs CPU-initiated perftest". Refuse and route
  elsewhere for general doca-gpunetio library work, DOCA install, the
  GPU-initiated WRITE latency analog, the CPU-initiated upstream
  perftest, or application-level end-to-end throughput — those belong
  to other skills.
metadata:
  kind: tool
compatibility: >
  Requires DOCA SDK on Linux with a BlueField DPU or ConnectX NIC,
  NVIDIA GPU, CUDA toolkit and nvcc, loaded `nvidia_peermem`, and an
  InfiniBand RNIC paired with the GPU. Uses `pkg-config` for
  doca-gpunetio, doca-rdma, and doca-common, plus the installed
  gpunetio_ib_write_bw sources. Run only on a trusted, non-shared IB
  fabric during the benchmark window.

DOCA GPUNetIO ib_write_bw

**Where to start:** This is a tool skill for the GPUNetIO- flavored `ib_write_bw` benchmark shipped under `doca/tools/gpunetio_ib_write_bw/` (a client + server pair, built from source against the installed DOCA via `meson`). It measures sustained RDMA WRITE bandwidth when the WRs are posted **from a CUDA kernel through the doca-gpunetio device-side surface**, with the GPU on the data path. Open [`TASKS.md`](TASKS.md) and start at [`## configure`](TASKS.md#configure) for the GPU-NIC pairing precondition and the build pattern; jump to [`## run`](TASKS.md#run) for the smoke-before-bulk flow. Open [`CAPABILITIES.md`](CAPABILITIES.md) when the question is *what this tool actually measures*, *how the result decomposes (GPU occupancy vs NIC issue rate vs link saturation)*, or *how the result reads against the GPI sister tool and the upstream CPU-initiated `perftest` `ib_write_bw`*. If DOCA is not installed yet, route to [`doca-setup`](../../doca-setup/SKILL.md) first; if the user is still deciding between the GPI and GPUNetIO programming surfaces, the picture in [`../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes`](../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes) and [`../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes`](../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes) is the first stop.

Example questions this skill answers well

The CLASSES of `doca-gpunetio-ib-write-bw` questions this skill is built to answer, each with one worked example. The class is the load-bearing piece; the worked example is one instance.

• **"What sustained RDMA-WRITE bandwidth can the GPUNetIO

path deliver on this GPU-NIC pair?"** — worked example: *"measure sustained WRITE BW between two hosts with an H100 + ConnectX-7 on each side"*. Answered by the GPU-NIC pairing precondition in [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)

• the bring-up flow in

[`TASKS.md ## configure`](TASKS.md#configure) + [`TASKS.md ## run`](TASKS.md#run). The same shape answers *"measure GPUNetIO-driven WRITE BW between a host GPU and a BlueField DPU"*.

• **"Where is the bottleneck — GPU compute occupancy, NIC

issue rate, or link saturation?"** — worked example: *"I see 120 Gbit/s on a 200 Gbit/s link; is the NIC saturated, am I CPU-bound on the client, or is the CUDA kernel not driving enough WRs in flight?"*. Answered by the throughput-decomposition rules in [`CAPABILITIES.md ## Observability`](CAPABILITIES.md#observability)

• the eval-loop overlay in

[`TASKS.md ## test`](TASKS.md#test).

• **"How does the result differ from the classic CPU-

initiated `perftest` `ib_write_bw`?"** — worked example: *"my team has a CPU-initiated WRITE BW number on this same NIC; should I expect the GPUNetIO number to match or be different?"*. Answered by the *"GPU-initiated path adds (or removes) overhead vs the CPU-initiated path"* rule in [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes).

• **"Is the doca-gpunetio path the right surface for my

sustained-throughput workload class?"** — worked example: *"my application streams sensor data from GPU memory at line rate to a remote consumer"*. Answered by the *"when GPUNetIO is the right surface vs GPI vs CPU- initiated"* rule in [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)

• the use-side decision in [`TASKS.md ## use`](TASKS.md#use).
• **"My BW number swings between runs. What do I check

before quoting it?"** — worked example: *"three runs at the same flags gave 145, 187, and 160 Gbit/s; is the benchmark noisy or is my platform inconsistent?"*. Answered by the measurement-soundness rules in [`CAPABILITIES.md ## Error taxonomy`](CAPABILITIES.md#error-taxonomy) layer 5 + the steady-state guidance in [`TASKS.md ## test`](TASKS.md#test).

• **"What version of DOCA + CUDA Toolkit do I need for this

binary to build and run?"** — worked example: *"my install has DOCA at one semver and CUDA at another; will the ToT-shipped `gpunetio_ib_write_bw` even link?"*. Answered by the version overlay in [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility) which cross-links the canonical detection chain in [`doca-version`](../../doca-version/SKILL.md).

Audience

This skill serves **external developers and performance engineers who need a