/doca-gpunetio

license: Apache-2.0
name: doca-gpunetio
description: >
  Use this skill when the user is doing hands-on DOCA GPUNetIO
  programming — wiring a CUDA kernel on an NVIDIA GPU to a doca-eth
  queue via doca_gpu_eth_rxq / doca_gpu_eth_txq, standing up the
  per-CUDA-device doca_gpu context, designing the persistent CUDA kernel
  that drains the GPU-visible queue, running the dual capability check
  (DOCA cap-query plus cudaGetDeviceProperties), registering cudaMalloc
  pools via doca_buf_arr_create_*, or debugging DOCA_ERROR_* returns
  from the GPUNetIO API. Trigger even when the user does not explicitly
  mention "DOCA GPUNetIO" or "persistent kernel" — typical implicit
  phrasings include "CUDA kernel reading packets directly from the NIC",
  "GPU-initiated networking on BlueField", "DOCA_ERROR_DRIVER on
  doca_gpu_create", "nvidia_peermem not loaded", "kernel-per-packet is
  too slow", or "which GPU supports GPU-side packet I/O". Refuse and
  route elsewhere for general CUDA programming, DOCA Ethernet queue
  bring-up, DOCA DPA, or DOCA install — those belong to other skills.
metadata:
  author: NVIDIA <doca-feedback@nvidia.com>
  kind: library
  doca_version: "3.5"
  tags:
    - doca
    - gpunetio
    - cuda
compatibility: >
  Requires DOCA SDK at /opt/mellanox/doca on Linux (Ubuntu 22.04/24.04
  or RHEL/SLES) with a BlueField DPU or ConnectX NIC. Reads the local
  install via `pkg-config doca-gpunetio`. Requires an NVIDIA GPU with
  CUDA toolkit (matched to DOCA per the DOCA Compatibility Policy) and
  GPU memory registered via dmabuf (or the legacy nvidia-peermem fallback) for GPUDirect RDMA; some
  samples need an InfiniBand-capable RNIC. The validation steps require
  a live DOCA install.

DOCA GPUNetIO

Examples

• **"CUDA kernel reading packets directly from the NIC."** — this is

in scope. Stay on `doca_gpu_eth_rxq` / `_txq` and the persistent kernel; do not rewrite it as host-CPU Ethernet.

• **"DOCA_ERROR_DRIVER on doca_gpu_create."** — check CUDA device

match and dmabuf / `nvidia-peermem` registration before changing queue type.

Instructions

Follow Loading order below. Open the tasks companion for configure / build / run. Open the capabilities companion for GPU vs host path selection. Do not open every companion file.

**Where to start:** This skill assumes DOCA is already installed, the CUDA toolkit is installed and matched to the DOCA install, and the user is doing **hands-on GPUNetIO work** — i.e. wiring a DOCA network queue into a CUDA kernel on an NVIDIA GPU. Open the tasks companion if the user wants to *do* something (configure / build / modify / run / test / debug); open the capabilities companion when the question is *what can GPUNetIO express* on this version + this GPU. If the user has not installed DOCA yet, route to `doca-setup/SKILL.md` first; if the user has not set up the underlying Ethernet RX/TX queues yet, that is a DOCA Ethernet question — route to `doca-eth/SKILL.md`.

Example questions this skill answers well

The CLASSES of GPUNetIO questions this skill is built to answer, each with one worked example. The agent should treat the *class* as the load-bearing piece — the worked example is a single instance.

• **"How do I get a CUDA kernel to receive packets directly from

the NIC?"** — worked example: *"persistent kernel on one GPU reads packets from a `doca_gpu_eth_rxq` built on top of a representor `doca_eth_rxq` and counts them per-flow"*. Answered by the persistent-kernel pattern in

Capabilities and modes

• the GPU-side bring-up workflow in

configure.

• **"Can I run GPUNetIO on this GPU?"** — worked example: *"my

host has one Ampere card and one Turing card; which one supports GPU-initiated networking?"*. Answered by the dual capability-discovery rule (DOCA cap-query AND `cudaGetDeviceProperties` against the CUDA device ordinal) in

Capabilities and modes

• the device-enumeration step in

configure.

• **"Why does my GPUNetIO setup fail with

`DOCA_ERROR_NOT_SUPPORTED` even though doca-eth came up fine?"** — worked example: *"`nvidia_peermem` is not loaded so GPUDirect RDMA is unavailable"*. Answered by the env preconditions in ## Safety policy

• the env checklist in

configure step 1.

• **"How do I move data between CUDA-allocated buffers and a DOCA

queue?"** — worked example: *"use `cudaMalloc` for the receive buffer pool and register it with DOCA via `doca_buf_arr_create_*` before starting the context"*. Answered by the CUDA-allocator

• DOCA-registration overlay in

Safety policy

• the buffer-prep step in

configure step 4.

• **"Is the GPUNetIO API I'm reading about on my installed DOCA +

CUDA combination?"** — worked example: *"is the persistent-kernel helper available with the CUDA toolkit version I have?"*. Answered by the version-compatibility overlay in

Version compatibility

which cross-links the canonical detection chain in `doca-version/SKILL.md` and adds the GPUNetIO-specific *DOCA must match CUDA* overlay.

• **"What does this `DOCA_ERROR_*` from a GPUNetIO call mean and

which layer caused it?"** — worked example: *"`DOCA_ERROR_DRIVER` on `doca_gpu_*_create` — is it DOCA, CUDA, or the underlying doca-eth queue?"*. Answered by the GPUNetIO overlay on the cross-library taxonomy in

Error taxonomy

• the layered ladder in

debug that escalates to

`doca-debug/SKILL.md`.

Audience

This skill serves **external developers building applications that consume the DOCA GPUNetIO library** — i.e., users whose code calls `doca_gpu_*` from host C/C++ to stand up the per-GPU context and the GPU-visible queue handles, and whose CUDA kernel (`.cu` translation unit) uses those handles from device code to submit / receive packets. The canonical target shape is the GPU Packet Processing reference application: a CUDA persistent kernel on an NVIDIA GPU that polls a GPU-visible RX queue and processes packets in-place on the GPU. It is