/meta-reviewing-ai-reviewing

name: meta-reviewing-ai-reviewing
description: AI integration review patterns. Use when reviewing model API calls, prompt construction, LLM output handling, RAG pipelines, and tool-calling code. Covers prompt-injection call-chain tracing, output validation, token budgets, retry/timeout handling, streaming, and key/PII exposure.

AI Code Review Patterns

> **Quick Guide:** When a diff touches model calls, trace every user-controlled string from its entry point into the prompt it lands in - that chain is the injection surface. Verify model output is validated before it drives control flow or storage, context growth is bounded, model calls carry timeouts and typed failure handling, and no keys or PII ride along in prompts or logs.

---

<critical_requirements>

CRITICAL: Before Reviewing AI Code

> **All code must follow project conventions in CLAUDE.md** (kebab-case, named exports, import ordering, `import type`, named constants)

**(You MUST trace every path where user-controlled input enters a prompt - through variables, retrieved documents, and tool results alike)**

**(You MUST verify every model response used in control flow, stored, or shown to other users is validated before use)**

**(You MUST check that conversation history and retrieved context are bounded - no accumulation without truncation)**

**(You MUST verify model calls carry a timeout and handle failure distinctly from success - transient errors retried or surfaced, never swallowed)**

**(You MUST verify no API keys, credentials, or PII appear in prompts, logs, or error messages the diff adds)**

</critical_requirements>

---

**Auto-detection:** review AI code, LLM PR review, prompt review, model call review, RAG review, tool calling review, agent loop review, completion handling review

**When to use:**

• Reviewing diffs containing model API calls (Anthropic, OpenAI, or other provider SDKs)
• Reviewing prompt construction, templates, or system prompt changes
• Reviewing parsing/consumption of LLM responses, including streaming
• Reviewing RAG retrieval pipelines or embedding code
• Reviewing agent orchestration and tool-calling loops

**When NOT to use:**

• When implementing AI features (use the relevant AI implementation skill)
• For plain REST/database code in the same diff (use the API reviewing skill)
• For prompt-quality tuning with no code change

**Key patterns covered:**

• Prompt-injection call-chain tracing
• Output validation before control flow and storage
• Token budget and context-growth review
• Failure handling around model calls (timeout, retry, typed errors)
• Streaming and partial-response handling
• Key/PII hygiene and model pinning

**Detailed Resources:**

• [examples/core.md](examples/core.md) - Good/bad AI integration patterns to look for during review

---

<philosophy>

Philosophy

**A model call is an untrusted boundary in both directions.** What goes in can carry an attacker's instructions; what comes out is a guess shaped like an answer. The review's job is to find the places the diff treats either direction as trusted.

**When reviewing AI code:**

• Map the call chain first: input entry → prompt assembly → API call → response parsing → consumption. Findings live at the joints.
• Treat retrieved documents and tool outputs as user-controlled - injection via a fetched page is still injection
• Ask of every response use: "what happens when the model returns garbage that parses?"
• Cost is a correctness concern: an unbounded loop of model calls is a defect even when every call succeeds

**When NOT to flag:**

• Don't demand fallback model chains for internal tooling or batch jobs where failing loudly is fine
• Don't demand prompt caching or cost optimization the spec never asked for
• Don't flag deterministic-output assumptions in throwaway scripts the way you would in a user-facing pipeline
• Don't demand eval harnesses in a diff that only rewords a prompt - note it and move on

**Core principles:**

• **The chain is the unit of review**: a sanitized entry point means nothing if a later join splices raw input
• **Validation before consumption**: a schema between the model and your control flow
• **Bounded everything**: history, retrieval, retries, and loops all need a ceiling
• **Failure is a first-class path**: model APIs fail often enough that unhandled rejection is a design decision, not an oversight

</philosophy>

---

<patterns>

Core Patterns

Pattern 1: Prompt-Injection Call-Chain Tracing

Follow every user-controlled string to the prompt it lands in.

## Injection Trace

For EACH prompt the diff constructs or changes:

- [ ] List every interpolated value and classify it: static, developer-controlled, or user-controlled
- [ ] User-controlled values are delimited/structured (tagged blocks, separate messages), not spliced into instructions
- [ ] Retrieved documents and tool results are treated as user-controlled
- [ ] System-prompt content (rules, secrets, tool definitions) is never echoed into user-visible output paths
- [ ] Instructions and data are separated - the prompt never asks the model to obey text it also lets the user write

// Must Fix: user text spliced into the instruction channel
const prompt = `You are a support agent. ${userMessage}. Use the refund tool if appropriate.`;

// Good: instructions and untrusted data are separate, and the data is fenced
const messages = [
  { role: "system", content: SUPPORT_AGENT_RULES },
  {
    role: "user",
    content: `<customer_message>${userMessage}</customer_message>`,
  },
];

**Why this matters:** Injection is a chain property, not a line property. The one splice you don't trace is the one that lets a pasted email operate your tools.

---

Pattern 2: Output Validation Before Consumption

A model response is a guess - validate it before it acts.

## Output Validation Review

For EACH place the diff consumes a model response:

- [ ] Structured output is schema