/gen-test-plan

description: Analyze repo, detect stack, trace changes to user-facing entry points, generate E2E YAML test plan
name: gen-test-plan
disable-model-invocation: true

Generate Test Plan

Analyze the repository's tech stack, branch changes vs default, and generate an executable YAML test plan focused on user-facing impact.

**This is an E2E test plan — not an automated test wrapper.** The generated plan will be executed by an autonomous agent acting exactly as a human QA tester would: launching real binaries, hitting real endpoints, interacting with real databases, and verifying real observable behavior.

Critical Rule: No Automated Test Duplication

**NEVER generate test steps that re-run the project's existing automated test suite.** This means:

• No `cargo test`, `pytest`, `npm test`, `go test`, `mix test`, or equivalent commands as test steps
• No wrapping unit/integration test modules in a test case
• No "run the tests and check they pass" — that's CI's job, not QA's

If you find yourself writing a test step that invokes the project's test runner, **stop and rethink**. Ask: "What would a human tester do to verify this feature works?" The answer is never "run the unit tests."

**What E2E test steps look like:**

• Build the binary and run it with real arguments, check stdout/stderr/exit code
• Start a server and hit it with curl
• Run a CLI command that writes to a real database, then query the database to verify
• Launch the TUI and verify it renders (via screenshot or process lifecycle)
• Chain multiple commands that exercise a full user workflow end-to-end

Hard gates

Complete these **in order**. Do not advance to the next gate until its **Pass** condition is met (each pass should leave retrievable evidence: pasted command output, a written list, or the generated file on disk). **Scheduling:** Gate **1** before Step **2**; Gate **2** before Step **5**; Gate **3** before Step **7**; Gates **4–5** during **Step 8** (after the Step 7 summary).

1. **Diff and base pinned (after Step 1)** — Resolve the base branch from `--base` when provided, otherwise use the repo default (`main` or `master` per Step 1). Compare `HEAD` to `$(git merge-base HEAD origin/<base_branch>)` (or equivalent if the remote ref differs). **Pass:** You record `current_branch`, `base_branch`, the merge-base SHA or range used, and `changed_files` from `git diff --name-only <merge-base>..HEAD` (empty list allowed if you paste or quote that output and state “no file changes vs base”).

2. **Trace complete (after Step 4)** — **Pass:** Every affected entry point you will test has a **Core functionality** vs **Configuration/admin** classification, and the Step 4 requirement holds: at least one test targets a core entry point **or** you document why that is impossible and flag manual review.

3. **Plan file valid (after Step 6, before Step 7)** — **Pass:** `docs/testing/test-plan.yaml` exists and the following command exits 0 (parses the YAML **and** asserts all four top-level keys are present — a single `grep -E` with alternations would pass on any one match, so do not substitute it):

    python3 -c "import sys, yaml; d = yaml.safe_load(open('docs/testing/test-plan.yaml')) or {}; missing = [k for k in ('version', 'metadata', 'setup', 'tests') if k not in d]; sys.exit('Missing keys: ' + ', '.join(missing) if missing else 0)"

4. **No automated-test duplication (Step 8)** — **Pass:** Every `run:` step and every `services:` `command:` is scanned for project test runners (`cargo test`, `pytest`, `npm test`, `go test`, `mix test`, `jest`, `vitest`, `mocha`, etc.); **zero** invocations. If any appear, remove or replace them with real E2E actions and re-run Gate 3.

5. **Behavioral coverage (Step 8)** — **Pass:** Re-read `metadata.changes_summary` and recent commit messages; at least one test’s `context`/`steps` exercises the primary user-visible behavior they describe. If they describe a capability (e.g., a new provider) but no step invokes it, add that test or fail verification.

Arguments

• `--base <branch>`: Base branch to diff against (default: `main`)
• Path: Target directory (default: current working directory)

Step 1: Gather Repository Context

# Get current branch
git rev-parse --abbrev-ref HEAD

# Resolve base branch: use --base if supplied, otherwise default (main → master)
BASE_BRANCH="${BASE_BRANCH:-$(git rev-parse --verify origin/main >/dev/null 2>&1 && echo main || echo master)}"
MERGE_BASE="$(git merge-base HEAD "origin/${BASE_BRANCH}")"

# Get changed files vs base
git diff --name-only "${MERGE_BASE}"..HEAD

# Get commit messages for context
git log --oneline "${MERGE_BASE}"..HEAD

**Capture:**

• `current_branch`: Branch name
• `base_branch`: Default branch to compare against
• `changed_files`: List of modified files
• `commit_messages`: What the PR is about

Step 2: Detect Tech Stack

See [references/stack-discovery.md](references/stack-discovery.md) for stack detection commands, entrypoint discovery, port discovery, and trace rules.

Step 3: Discover User-Facing Entry Points

A "user-facing entry point" is anything a human interacts with: CLI subcommands, HTTP endpoints, UI routes, TUI screens, gRPC services, database migrations, or configuration files that affect runtime behavior.

CLI Applications (Rust/clap, Python/argparse/click, Go/cobra)

# Rust (clap) — look for Subcommand derives and command enums
grep -rn "Subcommand\|#\[command\]" --include="*.rs" | head -20

# Python (click/typer/argparse)
grep -rn "@click.command\|@app.command\|add_parser\|add_subparser" --include="*.py" | head -20

# Go (cobra)
grep -rn "cobra.Command\|AddCommand" --include="*.go" | head -20

Build a map of:

• CLI subcommands: command name + description + file:line
• Required arguments and flags per subcommand
• Environment variables the binary reads (grep for `env`, `std::env::var`, `os.Getenv`, `os.environ`)

HTTP/API Services

**Python (FastAPI/Flas