/qa-strategy

name: qa-strategy
description: |
  Test-system design discipline for the QA route: choosing the right tier, designing
  scenario matrices with pipeline-wide observation points, test-environment topology
  and isolation, fixture lifecycle, and flake sources. Loaded by qa-researcher,
  qa-harness-builder, and qa-executor; read as a coverage lens by
  plan-gap-reviewer on QA plan reviews (it arrives as a file to Read in the
  task scaffold, not a SKILL_HINTS entry).
allowed-tools: Read Grep Glob Bash LSP
user-invocable: false

QA Strategy

**Status: DRAFT.** This skill is the QA route's test-system design discipline, and it is what `skills/building/references/integration-and-live-proof.md` sends a BUILD phase to for test-environment topology, observation points, and flake sources. Sections marked PLACEHOLDER need a deep dive.

**Core:** A test suite's value is not how many tests it has. It is how much you would believe a green run.

Tier selection

The three-layer model in `cc10x:building` still governs — unit proves local behavior, integration proves boundary wiring, E2E proves system truth. QA owns the outer two, plus UI.

| Tier | Proves | Use when | | ------ | -------- | ---------- | | `integration` | one boundary really works | route→service, service→DB, producer→consumer, cache/side effects | | `e2e_backend` | a flow across services really works | the value is produced by services cooperating | | `ui` | a human can actually do the thing | the deliverable is something a person operates |

**Choose the cheapest tier that can still fail for the right reason.** An E2E test that would also fail for ten unrelated reasons is a bad detector: it fires often and tells you little. Push a check down a tier whenever the lower tier can catch the same defect.

**But do not push everything down.** The defects that survive good unit coverage are exactly the ones that live *between* components — serialization mismatches, transaction boundaries, retry storms, partial failures. Those only appear at the tier where the components are real.

The scenario matrix

Every feature gets scenarios in three classes. A plan with only the first is not a test plan, it is a demo.

| Class | Question it answers | | ------- | --------------------- | | `happy-path` | Does the thing work when everything cooperates? | | `error-handling` | When a dependency fails, does the system fail *correctly* — right status, right message, right rollback, right log? | | `edge-case` | What about empty, one, many, max, concurrent, duplicate, out-of-order, expired, and unauthorized? |

Error-handling scenarios are where the bugs are

For each boundary the flow crosses, ask: what happens when it is slow, down, returns garbage, returns a partial result, or succeeds after the caller gave up? Each answer that matters is a scenario.

The most valuable single question: **when this fails halfway, what state is left behind?** Partial-failure state is where data corruption lives, and it is almost never covered by accident.

Input-space coverage — enumerate exhaustively, then reduce deliberately

The goal is to cover **as many user actions and chains of actions as possible**: every option in a dropdown, every date class, every filter, every combination that a real user could produce.

Step 1 — enumerate the full input space (do this exhaustively)

For every interactive surface in the flow, list every option a user can actually pick:

• every dropdown / select — **every** option, not "a representative one"
• every filter, toggle, checkbox, radio
• every date field — today, past, future, boundary of range, invalid, empty
• every free-text field — its equivalence classes
• every navigation path that reaches the same screen
• every chain — the ordered sequences of actions a user strings together

**Discover, do not assume.** Options are usually data-driven; reading the component tells you a `<select>` exists, not what is in it at runtime. Drive the real UI to enumerate actual values (see *UI tier tooling* below). An enumeration built from the code alone will miss the options that only appear for certain roles, tenants, or feature flags.

Record the full enumeration in the test plan even when you will not execute all of it. **The enumeration is the coverage claim.** What you skip must be visible.

Step 2 — reduce with a technique, not by taste

Full combinatorial coverage is not achievable and pretending otherwise produces an unbuildable plan: 5 filters × 4 options each is 1,024 combinations; add a date and a sort and you are past 10,000. Nobody runs that suite, so in practice it never gets written, and you end up with *less* coverage than an honest reduction would have given.

Reduce with a named technique, and **record which one you used and what it leaves uncovered**:

| Technique | Use for | What it costs | | ----------- | --------- | --------------- | | **Every-option-once** | dropdowns, radio groups | Each option exercised at least once. Misses interactions between options. | | **Pairwise (all-pairs)** | combinations of 3+ independent controls | Every *pair* of values co-occurs in some case. Collapses 1,024 → ~25. Misses 3-way interactions, which are rare. | | **Equivalence classes** | free text, numbers, dates | One representative per class of behavior. Misses defects inside a class. | | **Boundary values** | anything with a range | 0, 1, max, max+1, empty, null. Where most defects actually live. | | **Full combinatorial** | 2 controls with few values, or a genuinely safety-critical path | Complete. Only affordable when the space is tiny. |

**Default recipe:** every-option-once for single controls, pairwise across combinations, boundary values on every range, plus full combinatorial on any combination that is known to be risky (billing, permissions, anything where two settings interact by design).

**Pairwise is the highest-leverage tool here.** Most combination defects come from two settings interacting, not five. A