api-doc-verification-failures

API Documentation Verification Failures

<!-- no-pair-required: document introduction, not an individual failure mode block -->

> **Scope**: Detectable verification failures in API documentation — hallucinated params, untested examples, missing source verification. Does NOT cover style/structure standards (see `documentation-standards.md`). > **Version range**: REST APIs, OpenAPI 3.x, curl 7.x+ > **Generated**: 2026-04-15

---

Overview

API documentation fails in two modes: structural (bad formatting) and semantic (documenting the wrong thing). Semantic failures are harder to spot because they look correct. A parameter table with beautifully formatted rows for a field that doesn't exist in the source is a documentation defect, not a style issue. Every verification failure here is detectable by comparing the documentation against the source code or a running API.

---

Pattern Catalog

<!-- no-pair-required: section header with no content -->

Documenting Parameters Not in Source (Hallucinated Params)

**Detection**:

# Extract all documented parameter names from a doc file
grep -oP "(?<=\| )\w+" docs/api/endpoint.md | sort -u

# Then verify each one exists in the source
grep -rn "PARAM_NAME" src/ --include="*.go"
grep -rn "PARAM_NAME" src/ --include="*.py"
rg "PARAM_NAME" src/

**What it looks like**:

| user_id  | string | Yes | The user's unique identifier |
| metadata | object | No  | Optional metadata key-value pairs |

*(where `metadata` doesn't exist in the route handler)*

**Why wrong**: Integration failures happen silently. The caller sends `metadata`, the API ignores it, and the caller assumes it was accepted. Edge cases: some frameworks quietly drop unknown fields, others return 400. Either way, the doc is lying.

**Do instead:** Before writing any parameter, grep the source route handler for its exact name to confirm it exists. Zero grep results means the parameter is not real — remove it from the doc rather than publishing a lie.

**Fix**: Before writing any parameter, grep the source route handler for its exact name:

# For Go
grep -n "metadata\|user_id" handlers/users.go

# For Python/Flask
grep -n "request.json.get\|request.form.get" routes/users.py
rg "\.get\(['\"]metadata['\"]" src/

Zero results = parameter does not exist. Remove it from the doc.

---

Type Mismatches Between Doc and Source

**Detection**:

# Find integer params documented as string (common copy-paste error)
rg "int.*string|string.*int" docs/**/*.md

# Find the actual type in source (Go example)
grep -n "int\|string\|bool\|float" handlers/*.go | grep "PARAM_NAME"

**What it looks like**:

| page_size | string | No | Number of results per page. Default: 20 |

*(where the handler actually validates it as `int`)*

**Why wrong**: The caller sends `"50"` (string) and gets a 400. Or worse: the API coerces it silently and the caller never learns the real type. Type contracts are part of the API surface.

**Do instead:** Read the type from the validation struct or Pydantic model declaration, not from how the value is used in the handler. Document the type that the input binding layer enforces, which is the type the caller must send.

**Fix**: Read the validation code, not the handler call site:

# Find validation or binding in Go
grep -n "ShouldBindJSON\|ShouldBindQuery\|validate.Struct" handlers/*.go

# Find pydantic model in FastAPI
grep -n "class.*BaseModel\|: int\|: str\|: Optional" models/*.py

---

Untested curl Examples

**Detection** (check examples don't use placeholder values in actual requests):

# Find placeholder patterns in curl examples
grep -n "YOUR_TOKEN\|<token>\|example\.com\|placeholder" docs/**/*.md | grep "curl"
rg "Authorization: Bearer YOUR_" --glob "*.md"

**What it looks like**:

curl -X POST https://api.example.com/v1/users \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{"name": "John", "role": "admin"}'

*(where `role` was removed from the API last sprint)*

**Why wrong**: Copy-pasted examples become stale. If `role` was removed and the example still includes it, the API may return 400 and the new user thinks the docs are wrong — which they are.

**Do instead:** Run every curl example against a staging or test environment before publishing. Capture the actual response and confirm it matches the documented response example. If no live environment is available, add an explicit "verified against source at commit X" note.

**Fix**: Test the curl against a running service before publishing:

# Test with a real token against staging
TOKEN=$(cat .env | grep API_TOKEN | cut -d= -f2)
curl -X POST https://api-staging.example.com/v1/users \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "testuser"}' \
  -w "\nHTTP Status: %{http_code}\n"

If the test environment is unavailable, mark the example explicitly:

> **Note:** Example not verified against running service. Endpoint paths and parameters
> were verified against source as of commit `abc1234`.

---

Documented Error Codes Not Returned by Source

**Detection**:

# Find all error codes in docs
grep -oP "\b[45]\d\d\b" docs/api/endpoint.md | sort -u

# Verify each code is returned by source (Go example)
grep -rn "StatusBadRequest\|http.StatusUnauthorized\|400\|401\|403" handlers/endpoint.go

# Python/Flask
grep -rn "abort(400)\|abort(401)\|jsonify.*400\|make_response.*400" routes/

**What it looks like**:

| 418 | Teapot mode enabled | Disable teapot mode in config |

*(where the handler never returns 418)*

**Why wrong**: Readers write error handling code for 418. That code path is dead. The actual error they get from the API is 400 with an unhelpful message, and they have no documented path to resolution.

**Do instead:** Build the error table by grepping the handler file first, then documenting only the codes found. Start from