/model-compute-simulation

name: model-compute-simulation
description: "Build an operator-level compute template for an LLM and estimate FLOPs/MFU for a serving shape. Use when you need tensor shapes, per-op FLOPs, kernel-to-op MFU mapping, or parallelism what-if analysis."

Model Compute Simulation

Overview

Use this when the question is about operator order, tensor dimensions, FLOPs, MFU, or parallelism checks. The simulator loads a model config, builds the representative operator sequence, prints tensor shapes and FLOPs, and can estimate MFU from measured latency.

Inputs

Before running a simulation, collect or verify these inputs:

| Item | Why it matters | How to obtain | Default if user skips | |---|---|---|---| | Model name | Resolves to config in `model-config-index.json`; determines entire architecture | Ask user or infer from trace context | — (required) | | Config accuracy | Indexed values may differ from actual serving config (e.g. `routed_expert_intermediate_size`, `compress_ratios`) | Ask user to provide `config.json` or verify key params against HuggingFace | Use indexed values with a caveat | | GPU type | Determines peak FLOPS for MFU denominator | Ask user | — (required for MFU) | | dtype (bf16 / fp8) | Affects peak FLOPS selection; fp8 doubles peak | Ask user | bf16 | | Batch size & seq len | Directly affects FLOPs and tensor shapes | Ask user | B=1, S=1 (decode) | | TP / DP / EP | TP splits GEMM FLOPs across GPUs; EP splits expert FLOPs | Ask user | Read the run manifest; otherwise report assumptions explicitly | | Measured latency (ms) | Required for MFU numerator; must be per-GPU forward-pass wall-clock | Ask user or extract from a profiler trace | — (optional, no MFU without it) |

If the model is not in `model-config-index.json`, ask the user for a `config.json` path or add an indexed config before running estimates. The 2026-10-05 index pins public HF config revisions for V4 Flash/Pro/V4.1, K3, GLM-5.2/5.3/5.3-Flash, Qwen3.8-Flash-Next, Hy3/Hy4, Inkling, dots3, Step-3.7, Nemotron-3/3.5, LongCat and Gemma4, and corrects older model dimensions. `Qwen/Qwen3.8-A95B` returned HTTP 401 at this audit and is not indexed. Use `--config /path/to/config.json` for a raw nested HF config; the local normalizer expands text wrappers and expert/layer aliases.

Shape and timing limits for speculative MoE

For DSV4.1, obtain actual target/draft row counts from the trace: request BS=1 can issue six target-verify rows with DSPARK block size 5. Do not substitute acceptance length for the number of rows computed. Account for TP-sharded shared-expert shapes separately from routed expert assignments and EP. Do not divide an unverified model template by TP/EP and call it measured FLOPs. The simulator applies uniform TP division outside routed experts and uniform EP division inside routed experts. This is a what-if approximation: replicated routers, norms, latent projections and backend-dependent shared-expert ownership require a run-specific adjustment before interpreting MFU. The pinned V4.1 entry is a static target-model template; DSPARK draft, vision, Engram/PLE memory traffic and backend dispatch require separate evidence.

Kernel-duration sums include overlap and possibly PDL dependency waits. Use wall-clock time for a whole-pass MFU denominator, and measured GPU SKU/dtype peaks rather than assuming every FP8 operation doubles useful FLOPs. Follow the [dispatch and timing principles](../llm-torch-profiler-analysis/references/heuristics.md#establish-dispatch-and-numerical-contracts) when assigning kernel names to attention, shared experts, routing or mHC.

Workflow

Step 1: Load model config

Resolve the model name and load its configuration parameters:

python3 skills/model-compute-simulation/scripts/model_compute_simulator.py "<model name>" --list-models

The script resolves the model name against `references/model-config-index.json`, which stores public HuggingFace config parameters (hidden_size, num_experts, MLA ranks, etc.).

If the model is not indexed, tell the user to provide a `config.json` path or request an index update.

Step 2: Generate execution flow and tensor dimensions

Run the simulator with batch size, sequence length, and parallelism configuration:

python3 skills/model-compute-simulation/scripts/model_compute_simulator.py "Qwen3-235B-A22B" \
  --batch-size 1 --seq-len 1 \
  --tp 8 --dp 1 --ep 8 \
  --gpu b200 --dtype bf16

The simulator prints:

• Per-layer operator sequence with FLOPs and tensor shapes (shape_in → shape_out)
• Attention vs MoE/FFN FLOPs proportion per layer
• Total model FLOPs for a single forward pass

For **decode**: use `--seq-len 1 --context-len TOTAL_KV_LENGTH`. For extend, `--context-len` is the prefix length, with causal work for the new `--seq-len` tokens. Full/window attention includes KV context in QK and AV FLOPs. The LM head computes last-token logits unless `--all-token-logits` is given. Every run prints batch/shape/parallelism/GPU/dtype assumptions to stderr. Sparse selection, KDA/Mamba scalar work, norm/Sinkhorn and zero-expert top-k are architecture estimates, not measured instruction counts. LongCat routed FLOPs assume all top-k are nonzero (an upper bound until routing is measured). For **prefill**: use `--seq-len <prompt_length>`.

Step 3: Estimate MFU with measured latency

Provide the measured forward-pass latency to compute MFU:

python3 skills/model-compute-simulation/scripts/model_compute_simulator.py "Qwen3-235B-A22B" \
  --batch-size 1 --seq-len 1 \
  --tp 8 --dp 1 --ep 8 \
  --gpu b200 --dtype bf16 \
  --measured-ms 15.0

MFU = theoretical_min_time / measured_time × 100%

The simulator prints:

• Overall MFU
• Per-layer MFU (uniform layer-time assumption)
• Per-operator FLOPs proportion (for identifying which ops dominate)

GPU peak FLOPS are loaded from `references/gpu-specs.json`. The bundled hardware table includes H100/H200/B200, B300, GB300, MI355X, RTX PRO 6000 and DGX Sp