/design-testing-strategy

name: design-testing-strategy
description: Use before writing any type of tests. Distills 14 industry sources into deterministic decision gates, schemas, and worked test examples.

Design Testing Strategy

A reference manual for designing a fit-for-purpose, fit-for-criticality testing strategy.

This skill is **decision-oriented**, not philosophical: every gate is deterministic (ON when X / OFF when Y), every schema is enforced (field ordering matters), every example is worked end-to-end.

How To Use This Skill

1. Read **Decision Gates** in order (Gate 0 -> Gate 6). Each gate is independent — you may finish with any subset of test types ON. 2. Apply **Strategic Skip Heuristics** to remove ON gates that would yield low ROI for this artifact. 3. For each ON gate, fill the **Test Matrix Schema** (`selected_types` entry) — the field order is load-bearing. 4. List rejected types in `rejected_types` and deliberate skips in `deliberately_skipped`. 5. Produce a **Test Cases to Cover** markdown bullet list using ISTQB techniques from **Case Design Techniques**. 6. Cross-check against the matching **Worked Example** (A pure function / B HTTP+DB endpoint / C UI component).

---

Decision Gates

Apply gates in numeric order. Each gate produces an independent boolean (`applies: true|false`). Gates do NOT veto each other — a single artifact may have unit + integration + contract + property-based all ON.

| # | Type | ON when | OFF when | Source | |---|------|---------|----------|--------| | 0 | **Skip All** | Criticality is `NONE` (docs-only, comments, formatting, generated code, config without logic, throwaway prototypes) | Anything with branching, computed output, side effects, or user-visible behavior | [Pragmatic Programmer](https://pragprog.com/titles/tpp20/the-pragmatic-programmer-20th-anniversary-edition/) — "Test ruthlessly and effectively" implies effective skipping when ROI is zero | | 1 | **Unit** | Code contains any logic: branches, loops, conditionals, computation, transformation, parsing, validation, formatting | Pure declarative wiring (DI registration, route table) with no behavior | [Test Pyramid (Vocke)](https://martinfowler.com/articles/practical-test-pyramid.html) base layer + [Beck TDD](https://www.oreilly.com/library/view/test-driven-development/0321146530/) Red-Green-Refactor unit | | 2 | **Integration** | Boundary crossing: HTTP call, DB query, external SDK, message queue, filesystem I/O, OR collaboration with >=2 distinct collaborators where unit doubles distort behavior | Pure function with no I/O and 0-1 stable collaborators | [Testing Trophy (Dodds)](https://kentcdodds.com/blog/the-testing-trophy-and-testing-classifications) — integration is the highest-ROI layer; [Google "Follow the User"](https://testing.googleblog.com/2020/10/testing-on-toilet-testing-ui-logic.html) | | 3 | **Component or E2E** | UI surface AND criticality >= MEDIUM-HIGH AND user-facing critical path (signup, checkout, auth, payment, primary CTA) | Internal admin-only screens, dev tooling, or non-critical UI | [Test Pyramid top](https://martinfowler.com/articles/practical-test-pyramid.html) + [ISO/IEC/IEEE 29119](https://en.wikipedia.org/wiki/ISO/IEC_29119) risk ranking + [Google e2e principles](https://testing.googleblog.com/2016/09/testing-on-toilet-what-makes-good-end.html) | | 4 | **Contract** | Public API consumed by >=1 distinct clients (mobile + web, multiple internal services, external partners) AND independent deploy cadence | API where consumer and provider deploy together | [Pact / CDC](https://docs.pact.io/) + [Pactflow CDC explainer](https://pactflow.io/what-is-consumer-driven-contract-testing/) | | 5 | **Smoke** | Deployable surface (web app, API, service) AND a deploy/CI pipeline exists where post-deploy validation is meaningful | Library, internal helper, or no deploy pipeline | [Google "What Makes a Good End-to-End Test"](https://testing.googleblog.com/2016/09/testing-on-toilet-what-makes-good-end.html) — smoke = minimal e2e for deploy gate | | 6 | **Property-Based** | Input domain is large or unbounded (numeric ranges, strings, lists, parsers, serializers, encoders, math) AND invariants are stable (round-trip, idempotency, monotonicity, commutativity) AND criticality >= MEDIUM-HIGH | Small finite input domain, unstable invariants, or LOW criticality | [Hypothesis / QuickCheck](https://hypothesis.works/articles/what-is-property-based-testing/) |

Gate Application Algorithm

for gate in [Gate 0, Gate 1, ..., Gate 6]:
    if gate.ON_condition_met(artifact):
        result[gate.type] = applies: true
    else:
        result[gate.type] = applies: false

if Gate 0 is true:
    short-circuit: emit empty selected_types, document criticality=NONE, stop

**Criticality Scale** (used by Gates 3 and 6):

| Level | Definition | |-------|------------| | `NONE` | Docs, formatting, generated code, throwaway code, configs without logic | | `LOW` | Internal dev tooling, admin-only screens, logging formatters | | `MEDIUM` | Standard CRUD, internal APIs with a single team consumer, non-critical UI, helpers and utilities | | `MEDIUM-HIGH` | User-facing UI on critical paths, public APIs with multiple consumers, business workflows | | `HIGH` | Money movement, auth/authz decisions, security-critical validation, data integrity, regulated domains |

---

Test Type Reference

| Type | Use when | Do NOT use when | Frameworks | Typical dependencies | Google Size | |------|----------|-----------------|------------|----------------------|-------------| | **unit** | Pure logic, single function/method/class, deterministic inputs | Code is just I/O orchestration with no logic | vitest, jest, pytest, go test, JUnit, xUnit, RSpec | None (or in-memory fakes) | [Small](https://testing.googleblog.com/2010/12/test-sizes.html) | | **integration** | Boundary crossing (DB, HTTP, queue, FS); multiple collaborators where mocking distorts behavior | Pure function with no boundary | vitest, jest, pyt