/pt2-bug-basher

name: pt2-bug-basher
disable-model-invocation: true
description: Debug PyTorch 2 compiler stack failures including Dynamo graph breaks, Inductor codegen errors, AOTAutograd crashes, and accuracy mismatches. Use when encountering torch.compile errors, BackendCompilerFailed exceptions, recompilation issues, Triton kernel failures, FX graph problems, or when the user mentions debugging PT2, Dynamo, Inductor, or compiled model issues.

PT2 Bug Basher

Debug test failures and runtime errors in the PyTorch 2 compiler stack (Dynamo, Inductor, AOTAutograd, FX graphs).

Workflow Summary

1. **Environment check** -- Ask the user which conda environment to use. Verify it is active by checking `$CONDA_DEFAULT_ENV`. Then run `python -c "import torch; print(torch.__version__)"` to confirm torch is importable and report the version. If the environment is not active or torch cannot be imported, stop and ask the user to activate the correct environment before proceeding. 2. **Reproduce** -- Get a consistent reproduction of the failure 3. **Minimize** -- Reduce the repro to the smallest possible standalone case. Strip away unrelated model logic, use minimal tensor shapes, and isolate the specific op or pattern that triggers the bug. 4. **Add a unit test** -- **Do this BEFORE diving into code search or root cause investigation.** Add a failing test to the codebase that captures the bug. Place it in a specific, topic-appropriate test file (e.g., `test/dynamo/test_repros.py`, `test/inductor/test_torchinductor.py`, `test/export/test_export.py`). **Avoid `test/dynamo/test_misc.py`** — it is already oversized; find a more specific test file that matches the area of the bug. Use `torch.testing._internal.common_utils.TestCase` and `run_tests`. The test must fail before the fix and pass after. Having the test first keeps you grounded — you know exactly what "fixed" looks like before you start exploring the codebase. 5. **Validate on main** -- Use `EnterWorktree` to create a worktree checked out at `main`. Copy the new test file into the worktree and run the test there to confirm it **fails** on main. If the test passes on main, stop — the test may not be capturing the right bug, or the bug may already be fixed. Exit the worktree with `ExitWorktree` (action: remove) and return to the working branch before continuing. 6. **Gather logs** -- Run with appropriate `TORCH_LOGS` settings 7. **Classify** -- Use the [Error Triage](#error-triage) table to identify the category 8. **Inspect artifacts** -- Check FX graphs, IR, and generated code via `TORCH_COMPILE_DEBUG=1` 9. **Identify root cause** -- Trace from the error back through the compilation pipeline 10. **Fix** -- Apply the fix 11. **Verify** -- Run the new unit test AND nearby related existing tests (e.g., if you changed how `is_exporting` works, also run the existing `test_is_exporting` export test). Use `pytest -k` to quickly run related tests by name. The task is not complete until all pass. 12. **Self-review** -- Use the `/pr-review` skill to review your own changes before presenting them. Fix any issues it flags. 13. **Celebrate** -- Summarize the changes: explain the root cause, what was changed and why, and which tests were added/verified. Then tell the user the bug is squashed. Include a fun, varied motivational message or easter egg to keep spirits high (e.g., a pun, a quote, an ASCII art bug getting squashed). Keep it short and different each time.

Investigation Strategy

Prefer direct tools over meta_codesearch

Use `Grep`, `Glob`, and `Read` directly for code exploration. **Do not spawn `meta_codesearch` agents** — they are slow and expensive. The [Architectural Knowledge](#architectural-knowledge) and [Key Source Files](#key-source-files) sections below should give you enough context to know where to look. A targeted `Grep` for a function name is always faster.

Know which compilation mode you're in

Before reading implementation code, determine the compilation mode. These share code but diverge in important ways:

• **`torch.compile`** -- Dynamo + Inductor. `tx.export=False`, no `_compiling_state_context()`.
• **`torch.export` (strict)** -- `tx.export=True`, `_compiling_state_context()` active.
• **`torch.export` (non-strict, **the default**)** -- Uses Dynamo via `fullgraph_capture` but `tx.export` may differ from strict. `_compiling_state_context()` active. Check `torch._export.config.use_new_tracer_experimental` — it changes which code path is used.

Distinguish trace-time vs runtime

Many PT2 bugs come from confusing these two:

• **Trace-time**: Inside Dynamo's symbolic interpreter. Dynamo intercepts function calls and may constant-fold them (e.g., `is_exporting()` → `ConstantVariable(True)`).
• **Runtime**: Real tensors, real Python calls, module-level flags like `torch.compiler._is_exporting_flag`.

When debugging, add temporary `print()` statements directly in the source file rather than monkey-patching from outside — dispatch chains make monkey-patching unreliable.

Gathering Information

Pick the right diagnostic tool based on the error category:

• **Quick overview**: `TORCH_LOGS="+dynamo,graph_breaks,recompiles" python your_script.py`
• **Full debug artifacts**: `TORCH_COMPILE_DEBUG=1 python your_script.py` — creates `torch_compile_debug/` with FX graphs, Inductor IR, and generated code
• **Generated code only**: `TORCH_LOGS="output_code" python your_script.py`
• **Structured tracing**: `TORCH_TRACE=/path/to/trace python your_script.py` then `tlparse /path/to/trace`
• **Single-threaded (for pdb)**: `TORCHINDUCTOR_COMPILE_THREADS=1 python your_script.py`

Error Triage

Classify the failure using the error message and traceback:

| Error Pattern | Category | Jump To | |---|---|---| | `Unsupported: ...` or `graph break` in logs | Graph break | [Graph Breaks](#graph-breaks) | | `BackendCompilerFailed` | Inductor/backend crash | [Backend Failures](#backend-compiler-failures) | | `RecompileError` or `cache_size_li