/ml-debug

name: ml-debug
description: Use when something is failing in ML/AI work — OOM, NaN, divergence, crashes, bad throughput, wrong outputs, dependency conflicts

ML Debugging

Systematically diagnose ML failures using framework-specific knowledge, not guesswork.

Grounding

**Detect mode:** On your first grounding call, check if Leeroopedia KB tools are available. If they return results, use **KB mode**. If unavailable or auth fails, use **Web mode**.

**HARD RULE: You MUST ground before writing analysis.** If KB fails, you MUST WebFetch at least 2 URLs before writing ANY diagnosis. Writing from memory without fetching is the #1 failure mode of this skill — it produces zero-citation responses that score 0/3 on grounding. "I know X well" is NOT a substitute for fetching documentation.

**KB mode:** Call `diagnose_failure` → `query_hyperparameter_priors` → `search_knowledge`. Cite as `[PageID]`.

**KB mode grounding supplement (MANDATORY):** After KB calls, you MUST WebFetch at least 2 public URLs (official docs, PyPI, GitHub issues/source) and cite them as `[source](URL)` alongside `[PageID]` citations. KB-only responses score 2/3 max on grounding because reviewers cannot verify proprietary page IDs. Pattern: KB call for diagnosis → WebFetch PyPI for version → WebFetch GitHub/docs for config verification → cite BOTH KB and public URLs in every section. **Self-test**: if your response has zero `[source](URL)` citations, you will lose a grounding point regardless of KB citation count.

**Web mode:** WebFetch GitHub issues for the error message → WebFetch framework troubleshooting docs → WebFetch config references. Cite as `[source](URL)`. Start response with: `> Grounding: Web mode — citations from official docs and GitHub issues.`

**Web mode grounding targets by response section** (aim for these counts):

• Diagnosis root cause: 1+ citation (to a specific doc section or GitHub source line, NOT a top-level page)
• Each "Why it matters" explanation: 1+ citation or `[no KB]`
• Each fix step: 1+ citation for the specific API/config being changed
• Each quantitative claim ("X× faster"): 1 citation or `[no KB]`
• Prevention items: 1+ citation for the metric/tool referenced

Target: 5+ total citations in web mode, each linking to a specific doc section or source line (not top-level domain pages). Below 3 is a grounding failure. Generic page links (e.g., `huggingface.co/docs/transformers`) score lower than specific section links (e.g., `huggingface.co/docs/transformers/model_doc/mixtral#MixtralConfig`).

**PyPI pages are version-only citations**: A PyPI link confirms a version number but contributes zero technical content. Do NOT count PyPI links toward your citation minimum. You need 3+ citations that contain *technical claims* (API signatures, config defaults, known failure modes). PyPI fetches are a prerequisite step, not a citation source.

When citing GitHub source files, use a tagged release URL (e.g., `github.com/huggingface/transformers/blob/v4.45.0/src/...`) NOT the `main` branch. **MANDATORY first WebFetch in web mode**: fetch the framework's PyPI page (`https://pypi.org/project/<package>/`) to get the current stable version — then use that version tag in ALL subsequent GitHub URLs. **Do NOT fabricate version numbers** — if you haven't fetched the PyPI page, you don't know the current version. Writing "DeepSpeed 0.18.7" or "Transformers v5.3.0" without fetching is fabrication. **Multi-package rule**: If your diagnosis involves N frameworks, you need N PyPI fetches — one per package. Fetching DeepSpeed's version does NOT tell you the Transformers version. Each `**Version**:` line must link to the specific PyPI page it came from. If you cannot fetch any URLs, use the Ungrounded response format. **Self-test**: before writing your response, count your `[source](URL)` citations. If < 2, you have not met the minimum bar — fetch more or switch to Ungrounded format.

**Web mode response template** (use this structure when in web mode):

> Grounding: Web mode — citations from official docs and GitHub issues.
> Sources fetched: [URL1], [URL2]

## Diagnosis
**Root cause**: [one sentence] [source](URL)
**Version**: [exact version from fetched docs] [source](URL)
...

**Web mode URL registry:**

• PyTorch issues: `https://github.com/pytorch/pytorch/issues`
• HF Transformers issues: `https://github.com/huggingface/transformers/issues`
• DeepSpeed issues: `https://github.com/microsoft/DeepSpeed/issues`
• vLLM issues: `https://github.com/vllm-project/vllm/issues`
• PEFT docs: `https://huggingface.co/docs/peft`
• Axolotl issues: `https://github.com/axolotl-ai-cloud/axolotl/issues`
• vLLM docs: `https://docs.vllm.ai/en/latest/`
• PyTorch docs: `https://pytorch.org/docs/stable/`
• DeepSpeed docs: `https://www.deepspeed.ai/docs/config-json/`
• PyPI (version lookup): `https://pypi.org/project/<package>/` (use to find current stable version)
• HF Transformers releases: `https://github.com/huggingface/transformers/releases`
• vLLM releases: `https://github.com/vllm-project/vllm/releases`

The Iron Law

NO FIX WITHOUT UNDERSTANDING THE ROOT CAUSE FIRST

Applying fixes without diagnosis leads to fix-on-fix layering. The third "fix" usually breaks something the first fix was hiding.

NO CLAIM WITHOUT A [PageID] CITATION — OR MARK IT [no KB]

Every factual claim about framework internals requires either a `[PageID]` citation or an explicit `[no KB]` tag. There is no middle ground. "I have deep knowledge" is not a citation.

IF KB CALL FAILS → EVERY SECTION HEADER GETS ⚠️, EVERY CLAIM GETS [no KB], EVERY CONFIDENCE IS "Low"

This is an Iron Law, not a suggestion. A response that looks normal but has zero `[PageID]` and zero `[no KB]` tags is the #1 grounding failure.

NO WRITING FROM MEMORY — IF KB FAILS, YOU MUST WEBFETCH BEFORE WRITING

"I know X well" or "I have deep knowledge of X" followed by ungrounded analysis is the #1 anti-pattern. If KB fails: WebFetch docs