benchmark-agents
Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features —…
Full-story verification — infers what the user is building, then verifies the complete flow end-to-end: browser → API → data → response. Triggers on dev server start and 'why isn't this working' signals.
$ npx -y skills add vercel/vercel-plugin --skill verification --agent claude-codeHow it fires
How this skill gets triggered: by you, by Claude, or both.
/verificationContext preview
The summary Claude sees to decide when to auto-load this skill.
Full-story verification — infers what the user is building, then verifies the complete flow end-to-end: browser → API → data → response. Triggers on dev server start and 'why isn't this working' signals.
name: verification
description: "Full-story verification — infers what the user is building, then verifies the complete flow end-to-end: browser → API → data → response. Triggers on dev server start and 'why isn't this working' signals."
summary: "Verify full user story: browser + server + data flow + env"
metadata:
priority: 7
docs:
- "https://vercel.com/docs/projects/project-configuration"
sitemap: "https://vercel.com/sitemap.xml"
pathPatterns: []
bashPatterns:
- '\bnext\s+dev\b'
- '\bnpm\s+run\s+dev\b'
- '\bpnpm\s+dev\b'
- '\bbun\s+run\s+dev\b'
- '\byarn\s+dev\b'
- '\bvite\s*(dev)?\b'
- '\bvercel\s+dev\b'
- '\bastro\s+dev\b'
importPatterns: []
promptSignals:
phrases:
- "verify the flow"
- "verify everything works"
- "test the whole thing"
- "does it actually work"
- "check end to end"
- "end to end test"
- "why isn't it working right"
- "why doesn't it work"
- "it's not working correctly"
- "something's off"
- "not quite right"
- "almost works but"
- "works locally but"
- "verify the feature"
- "make sure it works"
- "full verification"
allOf:
- [verify, flow]
- [verify, works]
- [check, everything]
- [test, end, end]
- [not, working, right]
- [something, off]
- [almost, works]
- [make, sure, works]
anyOf:
- "verify"
- "verification"
- "end-to-end"
- "full flow"
- "works"
- "working"
noneOf:
- "unit test"
- "jest"
- "vitest"
- "playwright test"
- "cypress test"
minScore: 6
retrieval:
aliases:
- end to end test
- full stack verify
- flow test
- integration check
intents:
- verify full flow
- test end to end
- check if app works
- validate implementation
entities:
- browser
- API
- data flow
- end-to-end
- verification
chainTo:
-
pattern: 'process\.env\.\w+|NEXT_PUBLIC_\w+'
targetSkill: env-vars
message: 'Environment variable references detected during verification — loading Env Vars guidance for proper configuration, vercel env pull, and branch scoping.'
skipIfFileContains: 'vercel\s+env\s+pull|\.env\.local'
-
pattern: 'middleware\.(ts|js)|proxy\.(ts|js)|clerkMiddleware|NextResponse\.redirect'
targetSkill: routing-middleware
message: 'Middleware/proxy detected during verification — loading Routing Middleware guidance for request interception, auth checks, and proxy.ts migration.'
-
pattern: 'streamText\s*\(|generateText\s*\(|useChat\s*\('
targetSkill: ai-sdk
message: 'AI SDK calls detected during verification — loading AI SDK guidance for streaming, transport, and error handling patterns.'
skipIfFileContains: 'toUIMessageStreamResponse|DefaultChatTransport'You are a verification orchestrator. Your job is not to run a single check — it is to **infer the complete user story** being built and verify every boundary in the flow with evidence.
Your focus is the **end-to-end story**, not any single layer.
Before checking anything, determine **what is being built**:
1. Read recently edited files (check git diff or recent Write/Edit tool calls) 2. Identify the feature boundary: which routes, components, API endpoints, and data sources are involved 3. Scan `package.json` scripts, route structure (`app/` or `pages/`), and environment files (`.env*`) 4. State the story in one sentence: _"The user is building [X] which flows from [UI entry point] → [API route] → [data source] → [response rendering]"_
**Do not skip this step.** Every subsequent check must be anchored to the inferred story.
Gather the current state across all layers:
| Layer | How to check | What to capture | |-------|-------------|-----------------| | **Browser** | Open the relevant page, check console, take screenshots | Visual state, console errors, network failures | | **Server terminal** | Read the terminal output from the dev server process | Startup errors, request logs, compilation warnings | | **Runtime logs** | Run `vercel logs` (if deployed) or check server stdout | API response codes, error traces, timing | | **Environment** | Check `.env.local`, `vercel env ls`, compare expected vs actual | Missing vars, wrong values, production vs development mismatch |
Report what you find at each layer before proceeding. Use this reporting contract:
> **Checking**: [what you're looking at] > **Evidence**: [what you found — quote actual output] > **Next**: [what this means for the next step]
Trace the feature's data path from trigger to completion:
1. **UI trigger** — What user action initiates the flow? (button click, page load, form submit) 2. **Client → Server** — What request is made? Check the fetch/action call, verify the URL, method, and payload match the API route 3. **API route handler** — Read the route file. Does it handle the method? Does it validate input? Does it call the right service/database? 4. **External dependencies** — If the route calls a database, third-party API, or Vercel service (KV, Blob, Postgres, AI SDK): verify the client is initialized, credentials are present, and the call shape matches the SDK docs 5. **Response → UI** — Does the response format match what the client expects? Is error handling present on both sides?
At each boundary, check for these common breaks:
Comprehensive Vercel ecosystem plugin — relational knowledge graph, skills for every major product, specialized agents, and Vercel conventions. Turns any AI agent into a Vercel expert.
Repo: vercel-labs/vercel-plugin
Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features —…
End-to-end benchmark suite for vercel-plugin. Runs realistic projects through skill…
Run vercel-plugin eval scenarios in Vercel Sandboxes instead of local WezTerm panels.…
Create and launch benchmark test projects to exercise vercel-plugin skill injection across…
Audit vercel-plugin performance on real-world projects. Extracts tool calls from Claude Code…
Release vercel-plugin — run gates, bump version, generate artifacts, commit, and push. Use…