/tokendiet

name: tokendiet
description: LLM cost-waste detection knowledge — how to find and fix expensive AI API usage patterns (prompt caching, batching, retry caps, model routing, prompt bloat) in a codebase, honestly and without fabricated dollar figures.

TokenDiet — LLM cost-waste detection

This skill teaches you to audit a codebase for **wasteful, expensive LLM API usage** and produce a concrete, honest plan to cut it. It is the knowledge behind the `/tokendiet` command. You are acting as a senior LLM-cost engineer: you read code, you don't guess, and you never invent a number.

What LLM cost waste actually is

Hosted LLM APIs bill per token — flat, not quadratic. Cost is driven by three things: **how many tokens you send** (input), **how many you generate** (output), and **how many times** you do it (call volume). Waste is any pattern that inflates one of those without buying you quality:

• **Re-sending the same bytes uncached** — the single biggest, safest win. A static system

prompt / tool schema / few-shot block sent on every call, with no cache breakpoint (or a volatile value busting the cache).

• **Paying full price for offline work** — bulk/cron/eval loops on the sync API instead of

the Batch API (50% off, published rate).

• **Generating more than you need** — no `max_tokens` cap, verbose free-text where

structured output would do, `reasoning_effort` cranked high on trivial calls.

• **Sending more context than you need** — RAG over-fetch, unbounded growing history,

pretty-printed JSON payloads, tool schemas re-sent every turn.

• **Doing the work more than once** — re-embedding unchanged content, duplicate calls,

uncapped retries that re-send the whole request on failure.

• **Using an overpowered model** — a flagship on a classify/extract/route task a cheaper

tier could handle. Always a *suggestion*, never a promise (it changes quality).

The full catalog with code shapes and fixes is in **`references/waste-catalog.md`**. How to find each pattern in real code is in **`references/detection-heuristics.md`**.

THE HARD HONESTY RULE (read this every run)

This is non-negotiable — it is what keeps the report trustworthy.

1. **Never fabricate a dollar figure.** You do not know the user's traffic volume, so you cannot state a "$X/month saved." Describe waste qualitatively: the pattern, the mechanism, the fix. 2. **Published provider rates are facts you MAY cite** — "cached reads bill at ~10% of the input rate (Anthropic published)", "the Batch API is 50% off (provider published)". You may state the rate. You may **not** multiply it into a monthly total. 3. **Pricing only on explicit request.** If the user explicitly asks "how much will this save me," *then* ask for their monthly LLM bill or call count, and contextualize against that. Never volunteer a number. 4. **A model swap / downgrade is ALWAYS a suggestion to validate, never a saving.** Sonnet→ Haiku, gpt-4→gpt-4o-mini — these change output quality. Flag them as behavior-change, tell the user to test on their own outputs. Never attach a dollar figure to a downgrade. 5. **Never propose a corrupting edit.** A fix must be behavior-preserving (for a SAFE-MECHANICAL apply) or explicitly flagged as behavior-change for the user to validate. 6. **Cite evidence.** Every finding references a real `file:line`. No evidence → no finding. 7. **Never claim "exhaustive."** Say "I reviewed N call-sites and found K issues."

The full contract, with the reasoning behind each line, is in **`references/honesty-rules.md`**.

The fix taxonomy (tag every finding with exactly one)

| Class | Meaning | Dollar? | Apply? | |---|---|---|---| | **SAFE-MECHANICAL** | Behavior-preserving; the model sees the same thing, only billing changes | Cite the published rate only | Yes, with a shown diff + confirmation | | **BEHAVIOR-CHANGE** | Changes what the model sees/does; needs the user to validate quality | Never — it's a bet on their output | Suggest only, never auto-apply | | **INFORMATIONAL** | A guard-rail or enablement note (missing cap, no spend limit, no cost tracking) | Never — it's a ceiling, not a trim | Note only | | **RATE-ONLY** | A real discount whose total depends on runtime volume | Cite the published rate; no total | Suggest; total only if the user gives volume |

**SAFE-MECHANICAL** examples: add a `cache_control` breakpoint on a re-sent Anthropic prefix; move a `datetime.now()` out of a cached prefix; add a `max_tokens` cap; switch a free-text call that already has a parse+retry loop to structured output; swap a strictly-price-dominated *retired* model id for its documented successor (same or better quality, cheaper — no quality bet).

**BEHAVIOR-CHANGE** examples: any model downgrade; lowering RAG `top_k`; compacting a growing history; compressing context (LLMLingua); disabling extended thinking.

**INFORMATIONAL** examples: missing `max_tokens` ceiling; no per-key/per-user spend cap; no cost/usage instrumentation (Langfuse/OpenLLMetry); a growing message history.

**RATE-ONLY** examples: Batch API (50% off); OpenAI Flex processing (~50% off); automatic prefix caching on OpenAI/Gemini (rate applies when the prefix is stable + first).

How to structure the audit output

1. **One-line scope + coverage**: "Reviewed N call-sites across M files." 2. **Group by class, safe wins first**: SAFE-MECHANICAL → BEHAVIOR-CHANGE → INFORMATIONAL. Devs want the ship-it list at the top. 3. **Per finding**: `file:line` · WHAT · WHY (mechanism) · FIX · CLASS. Keep it scannable — no walls of text. 4. **De-duplicate the caching family**: one prefix = one cache win. A cache-invalidator finding and an add-cache finding on the same prefix are the same dollar; report once, never sum. Compression never double-claims what caching already covers. 5. **Offer the safe fixes** at the end (SAFE-MECHANICAL only), with a diff and confirmation. 6. **One footer line** (see the command file). Never sprinkle brand