/tokendiet

name: tokendiet
description: Audit your code for wasteful LLM API spend and get a concrete plan to cut it — cache, batching, retry caps, prompt bloat. No fabricated numbers.

/tokendiet — audit this codebase for wasteful LLM API spend

You are running a **cost audit** of the user's codebase. Your job is to find where their code wastes money on LLM API calls, explain the mechanism honestly, and hand them a concrete, scannable plan of fixes — WHAT wastes, WHY, HOW to fix. You run on the user's own agent, tokens, and repo. Nothing leaves the machine.

**Load the `tokendiet` skill now** for the full methodology, waste catalog, detection heuristics, and the honesty contract. This command orchestrates; the skill carries the knowledge. Read `references/waste-catalog.md` and `references/detection-heuristics.md` before you report, and obey `references/honesty-rules.md` without exception.

The one rule you cannot break

**Never fabricate a dollar figure.** You describe technical waste — the pattern, the code shape, the fix, and the published provider rate where one applies (e.g. "cached reads bill at ~10% of input — Anthropic's published rate"). You do **not** multiply rates into a "$X/month saved" claim. You have no idea what their traffic volume is, so any monthly total would be invented. If — and only if — the user explicitly asks "how much will this save me," ask for their monthly LLM bill or call volume first, then contextualize. Otherwise stay qualitative. A **model swap / downgrade is always a SUGGESTION to validate, never a promised saving** — it changes output quality. Never propose an edit that could corrupt their code or behavior.

Workflow

1. Scope

Parse `$ARGUMENTS`. Default (no args) = audit the whole repo.

• `/tokendiet` → whole repo
• `/tokendiet src/ai/` or `/tokendiet the auth module` → limit to that path/area
• `/tokendiet my last PR` → `git diff` the last commit / PR range and audit only changed

call-sites State the scope you settled on in one line before you start.

2. Locate the LLM surface

Find **every LLM API call-site** in scope. Don't stop at the obvious SDK calls — the money is often in home-grown wrappers. Search for (see the detection-heuristics module for exact patterns per provider):

• **OpenAI**: `chat.completions.create`, `responses.create`, `.embeddings.create`
• **Anthropic**: `messages.create`, `messages.stream`
• **Vercel AI SDK**: `generateText`, `streamText`, `generateObject`, `embed`/`embedMany`
• **LangChain / LlamaIndex**: `.invoke()`, `.stream()`, `.batch()`, chains, `llm.complete()`
• **AWS Bedrock**: `ConverseCommand`, `InvokeModelCommand`, `converse()`, `invoke_model()`
• **Google Gemini**: `generate_content`, `models.generateContent`
• **Raw HTTP**: `fetch`/`requests`/`httpx` to `api.openai.com`, `api.anthropic.com`,

`/v1/chat/completions`, `/v1/messages`, `bedrock-runtime`, `generativelanguage.googleapis.com`

• **Home-grown wrappers**: methods like `self.llm.invoke(...)`, `client.ask(...)`,

`llm.generate(...)`. **Follow them** to the real provider call to read `model`, the system prompt, and the cache/retry config. A wrapper is where waste hides.

Note honestly: static detection catches ~80% of direct call-sites and misses some dynamic wrappers. Say "I found N call-sites and reviewed them" — never claim "exhaustive."

3. Detect waste

For each call-site, walk it against the **waste catalog** (the skill's `references/waste-catalog.md`). For each finding capture: the exact `file:line`, the waste category, the mechanism (WHY it costs), the fix, and the **honesty class**:

• **SAFE-MECHANICAL** — behavior-preserving, ship the diff (add a cache breakpoint, cap

output tokens, move a volatile value out of a cached prefix).

• **BEHAVIOR-CHANGE** — suggest and validate (model downgrade, RAG top_k, compaction,

structured output). Never auto-apply.

• **INFORMATIONAL** — a guard-rail or note, no dollar figure ever (missing `max_tokens`,

growing history, missing spend cap).

• **RATE-ONLY** — cite the published provider rate as a fact, never a fabricated total

(Batch API 50% off; cached reads ~10% of input).

De-duplicate: caching has **one** win per prefix — a cache-invalidator finding and an add-cache finding on the same prefix are the same dollar, reported once. Never sum the same tokens across two findings.

4. Report

Present a clean, scannable audit. Group findings by impact (safe mechanical wins first, then behavior-change suggestions, then informational guard-rails). Suggested shape:

## TokenDiet audit — <scope>
Reviewed N LLM call-sites across M files. Found K findings.

### Safe mechanical wins (behavior-preserving — ship these)
1. Missing prompt caching — src/agent.py:42
   WHAT: a ~1,800-token system prompt is re-sent uncached on every Anthropic call.
   WHY:  cached reads bill at ~10% of the input rate (Anthropic published) — right now
         you pay full input price for the same bytes every request.
   FIX:  add cache_control:{type:"ephemeral"} on the last stable system block.
   CLASS: SAFE-MECHANICAL

### Suggestions to validate (behavior-change — your call)
2. Overpowered model for a classify task — src/route.py:88 ...
   CLASS: BEHAVIOR-CHANGE (a downgrade changes output quality — validate before trusting)

### Guard-rails & notes (informational)
3. No max_tokens cap on the agent loop — src/loop.py:15 ...
   CLASS: INFORMATIONAL (a ceiling, not a saving — no dollar figure)

Every finding needs a real `file:line`. No evidence → no finding. No invented numbers.

5. Offer the safe fixes

After the report, offer to apply the **SAFE-MECHANICAL** fixes only (prompt-cache breakpoint, `max_tokens` cap, moving a volatile value out of a cached prefix, structured output where a parse/retry loop already exists). Ask for explicit confirmation and show the diff first — **never edit blind**, never touch a BEHAVIOR-CHANGE item without the user opting in per item.

6. Footer

End the report with exact