/qa

name: qa
description: "QA test your code changes by reading your git diff, choosing the right validation path for frontend/browser and backend changes, and reporting pass/fail with evidence."

QA — Validate Frontend and Backend Changes

Read the diff, classify what changed, and run the right validation path: browser QA for frontend/browser changes, API validation for backend surface changes, repo-native validation for backend-internal changes, and both for mixed changes.

<!-- NOTE: .agents/skills/qa/SKILL.md is the repository canonical source. Keep both synchronized copies in sync with it: 1. skyvern/cli/skills/qa/SKILL.md (bundled with the pip package) 2. skyvern/cli/mcp_tools/prompts.py (QA_TEST_CONTENT for the MCP prompt) -->

You changed code. This skill is diff-driven first: it reads what changed, understands the affected behavior, and validates that behavior with the right tools. It is not a generic website crawler, and it should not invent random API checks that are unrelated to the diff.

Quick Start

/qa                              # Diff-based: choose the right validation path automatically
/qa http://localhost:3000        # Same, explicit frontend URL
/qa -- validate the workflow filters API

How It Works

1. Read the code changes from `git diff` 2. Read the changed files to understand behavior, routes, schemas, and UI 3. Classify the diff as `frontend/browser`, `backend API`, `backend-internal`, or `mixed` 4. Run the right validation flow 5. Report pass/fail with concrete evidence

Step 1: Understand the Changes

Get the diff

# What files changed?
git diff --name-only HEAD~1     # vs last commit (if changes are committed)
git diff --name-only            # vs working tree (if uncommitted)

# Full diff for context
git diff HEAD~1                 # or git diff for uncommitted

Pick whichever diff has content. If both are empty, there is nothing diff-driven to QA.

Read the changed files

Read the full contents of every changed file that affects behavior:

• Frontend files: `.tsx`, `.jsx`, `.ts`, `.js`, `.css`, `.html`
• Backend/API files: routes, controllers, request/response schemas, serializers, handlers
• Backend-internal files: services, workers, business logic, validators, data-layer code
• Tests that changed alongside the implementation

Look for:

• route paths and page entry points
• component names, visible text, forms, buttons, error states
• API endpoints, request params, response fields, auth requirements
• validation logic, branching behavior, feature flags, empty states
• tests that describe the expected behavior

Step 2: Classify the Diff

| Mode | Trigger | Primary validation | |------|---------|--------------------| | Frontend/browser | UI/routes/components/styles changed | Browser QA against the dev server | | Backend API | Route handlers, request/response schemas, or externally visible API behavior changed | Start backend locally and run targeted API requests | | Backend-internal | Services/workers/business logic changed without public API surface changes | Repo-native fast checks plus targeted tests | | Mixed | Frontend/browser and backend changed together | Backend validation first, then frontend/browser QA |

Use these rules:

• If both frontend/browser and backend changed, treat it as `Mixed`.
• If only backend internals changed, do not invent unrelated browser tests or random API calls.
• If a backend change might affect the public contract, inspect routes, schemas, and tests before choosing `backend-internal`.
• If the diff is mostly documentation or comments, keep QA lightweight and report that no behavioral validation was warranted.

Step 3: Choose the Validation Strategy

Frontend/browser mode

Use browser automation against the dev server. Validate the specific UI changes plus 1-2 adjacent regression checks.

Backend API mode

Use the repo's documented local startup and auth instructions, start the backend if needed, identify the changed endpoint(s), and run targeted HTTP requests to validate the changed contract.

Backend-internal mode

Run the repo's fast verification commands first, then targeted unit/integration/scenario tests for the changed logic. Only start the backend and do live API calls if the change affects exposed behavior.

Mixed mode

Validate the backend first, then run frontend/browser QA against the flow that depends on it. If the backend contract is broken, frontend results are not trustworthy.

Step 4A: Frontend/Browser QA

Find the dev server

If the user provided a URL, use it. Otherwise auto-detect common local ports:

5173, 3000, 3001, 8080, 8000, 4200

If none respond, start the most direct repo-documented local command for the changed surface. If the diff needs both frontend and backend running together and the repo provides a combined frontend/backend dev script, prefer that. Only ask the user to start something manually if the repo has no documented command or startup fails.

Connect to a browser

Try these in order:

Option A: Local browser (fastest)

skyvern_browser_session_create(local=true, headless=false, timeout=15)

Use `local=true` so the browser can reach `localhost`.

Option B: Local browser via tunnel

If local session creation fails because the MCP server is remote, the cloud browser cannot reach `localhost`. Tell the user to run:

# Terminal 1: Launch a local browser with CDP exposed
skyvern browser serve --port 9222

# Terminal 2: Tunnel it to the internet
ngrok http 9222

Then connect:

skyvern_browser_session_connect(cdp_url="wss://<ngrok-subdomain>.ngrok-free.app/devtools/browser/<id>")

The user can get the browser ID from the `skyvern browser serve` output or by calling the ngrok URL's `/json` endpoint.

Option C: Cloud browser

skyvern_browser_session_create(timeout=15)

Only works for publicly reachable URLs. `localhost` URLs w