/llm-torch-profiler-analysis

name: llm-torch-profiler-analysis
description: "Analyze existing Torch Profiler Chrome traces or capture from SGLang, vLLM, TensorRT-LLM and TokenSpeed servers. Return kernel attribution, overlap opportunities and fusion patterns; use for model dispatch, CUDA Graph gaps, stream contention or exposed kernel tails. Includes existing SGLang Omni traces."

LLM Torch Profiler Analysis

Produce three tables: kernel/source attribution, overlap opportunities, and fusion patterns. Start from the user's trace or running server. Inspect the actual model, framework revision, phase, device/rank and parallelism before choosing a fast path. Existing traces need no GPU or framework installation. The Python analyzers use the standard library.

Read [source contracts](../../docs/upstream-source-contracts.md) for dated source inspections and version-sensitive profiler APIs. Apply the general [dispatch and numerical principles](references/heuristics.md#establish-dispatch-and-numerical-contracts) to every model. The [fusion catalog](references/fuse-overlap-catalog.md) contains concrete operator families, eligibility examples and storage contracts; the [overlap catalog](references/overlap-catalog.md) covers scheduling. Consult the model-history knowledge base when detailed PR evidence is needed. Source checks and historical captures are not fresh GPU validation.

Choose the evidence

| Mode | Evidence and limits | |---|---| | Existing trace | All four frameworks; SGLang Omni Python roots are also recognized. Use Chrome JSON or gzip with GPU events. A generic trace may need `--framework` explicitly. | | Single live capture | Use a shared server/client output directory and a supported HTTP profiler. Save actual launch arguments and package revisions. | | Mapping + formal | Eager/graph-off trace supplies call-site context; warmed graph trace supplies timing. Match signatures, shapes and stage before transferring attribution. Never transfer eager timing to the formal run. |

For a diffusion workload, record whether it uses the framework's native backend or a Diffusers fallback. A fallback trace can be analyzed, but does not establish native-kernel performance or dispatch.

Read a trace

Commands below are relative to this skill directory.

python3 scripts/analyze_llm_torch_profile.py \
  --framework sglang --input /path/to/TP-0-DECODE.trace.json.gz

python3 scripts/analyze_llm_torch_profile.py \
  --framework vllm \
  --mapping-input /path/to/eager-mapping \
  --formal-input /path/to/warmed-graph

Do not collapse a distributed run to rank0 when investigating rank skew or collective waits. Keep the original capture; a GPU-only or compact export is for sharing, not replacement evidence. Use [torch-profiler-layer-track](../torch-profiler-layer-track/SKILL.md) for verified L0/L1 guides and synthetic display lanes, or [llm-pipeline-analysis](../llm-pipeline-analysis/SKILL.md) for layer intervals.

Capture against a running server

The common helper defaults to separately driven prefill and decode workloads: nominal input/output lengths 4090/1 and 1/2048, with 10 warmup steps and 5 requested capture steps. Override these to match the slow serving scenario. Synthetic prompt lengths are not a tokenizer-independent guarantee; save actual usage and batch/graph dimensions. A long-context decode bottleneck needs a representative KV length, not automatically the one-token-input default.

python3 scripts/analyze_llm_torch_profile.py \
  --framework sglang --url http://127.0.0.1:30000 \
  --output-dir /shared/profiles/run-001 \
  --profile-workload both --warmup-steps 10 --num-steps 5 \
  --prefill-input-len 4090 --prefill-output-len 1 \
  --decode-input-len 4090 --decode-output-len 2048

`--profile-workload` accepts `both`, `prefill`, `decode`, or `legacy` (explicit probe prompt/token settings). Workload labels alone do not make a trace purely decode: inspect stage annotations and exclude request prefill when needed. The helper targets SGLang profiler v1 (default). Declare `--sglang-profile-v2` if the remote server enables `SGLANG_PROFILE_V2=1`; the helper refuses this incompatible flow before issuing control requests. `/server_info` does not expose remote environment variables. Native v2 needs `profile_by_stage:true` and `num_steps`, without `start_step`, `merge_profiles` or `/stop_profile`. The Rust frontend has no HTTP profiler routes; use Python or offline capture. The host launcher explicitly sets `SGLANG_PROFILE_V2=0`. The helper puts its label in `profile_id`, so merging can discover rank files. SGLang can emit stage-specific traces; its scheduler stop counter may require a final forward to close the window. The helper's SGLang workload adds a guard request/step. Check actual trace counts instead of assuming identical windows across frameworks.

For other frameworks add `--no-profile-by-stage`. Their HTTP windows and iteration controls differ:

• **vLLM:** launch with

`--profiler-config '{"profiler":"torch","torch_profiler_dir":"/shared/profiles/run-001","max_iterations":5,"ignore_frontend":true}'`. Start/stop has no per-request output/steps body. Configure iteration limits and optional profiler warmup at server launch. The helper's `--num-steps` controls probe planning, not vLLM's internal profile schedule. `torch_profiler_with_stack` defaults true in the inspected source. Frontend traces can contain no kernels; select GPU-worker traces for GPU analysis. `active_iterations` only takes effect with a warmup/wait schedule; use `max_iterations` to bound workers. The host option is `--profiler-max-iterations` (`--profiler-active-iterations` remains an alias). Two capture rounds require [vLLM #57460](https://github.com/vllm-project/vllm/pull/57460), merged 2026-09-21; older images need separate launches for prefill and decode. Keep CPU in `torch_profiler_activities` for source attribution and step annotations; CUDA-only activities omit both Python stacks an