/meta-reviewing-ai-reviewing

name: meta-reviewing-ai-reviewing
description: AI integration review patterns. Use when reviewing model API calls, prompt construction, LLM output handling, RAG pipelines, and tool-calling code. Covers prompt-injection call-chain tracing, output validation, stop reasons, token budgets, retry and timeout handling, streaming, and key/PII exposure.

AI Code Review Patterns

> **Quick Guide:** A model call is untrusted in both directions, so the review follows every user-controlled string into the prompt it lands in, and every response out to whatever it drives. Four facts about current providers change what counts as a defect. The Anthropic and OpenAI SDKs retry twice and time out after ten minutes by default, so a call with no options is protected but too patient for a person waiting, and a retry wrapper around it multiplies attempts. A response cut off at the token limit, or refused, arrives as a successful call. A stream can fail after its 200. And current reasoning models reject a non-default `temperature`, so setting one buys a 400 rather than determinism.

**Detailed Resources:**

• [examples/core.md](examples/core.md) — the defect and its fix for each pattern below
• [reference.md](reference.md) — per-provider lookup: SDK retry and timeout defaults, stop and finish reasons, structured-output parameters, sampling parameters, history rules

---

<critical_requirements>

Before reviewing AI code

**Trace every value interpolated into a prompt back to where it entered — retrieved documents and tool results included.** Injection is a property of the chain rather than the line: input sanitised at the entry point and spliced raw at a later join is still injected, and a fetched page is as hostile as anything a user types.

**Find what each model response is allowed to do, and check it is validated before it does it.** A response that picks a handler, fills tool arguments, is stored, or is shown to another user is untrusted input; models return malformed and adversarial output at some rate forever, and the check between response and action decides what that rate costs.

**Find the ceiling on everything that grows — history, retrieved context, retries, and loops that call the model.** Every token is billed on every call, so an unbounded one costs more each turn until it fails at the context limit, first for the most engaged user.

**Establish the timeout and retry policy each call actually runs with, SDK defaults included, and check failure reaches the caller as failure.** A call with no options gets ten minutes and two retries, timeouts included, so it can hang for half an hour; and a failure swallowed into an empty answer reads downstream as the model having nothing to say.

**Check that keys stay out of prompts and client bundles, and user content out of logs and error messages.** Anyone who talks to the model can extract its system prompt, and one debug line that logs a prompt ships every user's message to wherever logs go.

</critical_requirements>

---

**Auto-detection:** AI code review, LLM PR review, `messages.create`, `messages.stream`, `chat.completions.create`, `responses.create`, `stop_reason`, `finish_reason`, `max_tokens`, `maxRetries`, `tool_use`, `tool_calls`, `output_config`, `response_format`, system prompt, prompt template, retrieved chunks, embeddings, agent loop

**Applies to:**

• Diffs that add or change a model call, its options, or the client it runs on
• Prompt assembly — templates, system prompts, retrieved context, tool results fed back
• Code that consumes a response — parsing, tool dispatch, storage, display, streaming
• Agent and tool-calling loops
• Grading the AI-specific findings by severity

**Handled elsewhere:**

• Building the integration — choosing the provider and writing the prompts and client code is implementation; this skill finds what the diff gets wrong.
• Budgets, eval thresholds and the output contract — decided when the feature is specified; review checks the diff against whatever was decided.
• The code around the call — the handler, the database write and the auth check are reviewed as that layer's code; this skill covers the model boundary.
• Prompt wording with no code change — whether an instruction works is answered by an eval, not by reading the diff.
• Running and writing up the review — reading order, file:line evidence, tone and the approval decision belong to the general review process.

---

<philosophy>

Philosophy

**A model call is an untrusted boundary in both directions.** What goes in can carry an attacker's instructions; what comes out is a guess shaped like an answer. The review finds the places the diff treats either direction as trusted.

Map the chain before reading lines: input entry → prompt assembly → API call → response checks → consumption. Findings live at the joints — the fetch was fine, the embedding was fine, and the splice between them is where the injection lands.

Ask of every response use what happens when the model returns garbage that parses — a valid label that is the wrong one, a well-formed refund for the wrong amount. Every check in Pattern 2 passes it, because a schema checks shape and never truth.

Cost is a correctness concern. An unbounded loop of model calls is a defect even when every call succeeds.

**Leave these unflagged:**

• Fallback model chains for internal tooling or batch jobs, where failing loudly is fine
• Prompt caching or cost optimisation the spec never asked for
• A missing `temperature: 0` — current reasoning models reject it, and where it is accepted it does not make output repeatable
• Eval harnesses in a diff that only rewords a prompt — note it and move on
• Throwaway scripts held to the standard of a user-facing pipeline

</philosophy>

---

<decision_framework>

Severity

Is it a safety or correctness defect the diff introduces?
├─ User-controlled text spliced into the instruction channel → MUST FIX
├─ Private data + untrusted input + an outbound or mut