/verification

name: verification
description: |
  Use when judging whether a task reached its goal, not just finished: the gate function,
  self-critique gate, validation levels, evidence array protocol, and goal-backward lens.
  Task completion is not goal achievement.
allowed-tools: Read Bash Grep Glob
user-invocable: false

Verification

**Task completion is not goal achievement.** Verify that the phase achieved its goal, not that prior agents said it did. If you cannot independently reproduce a claimed success, return FAIL.

Reference Files

• `references/live-production-testing.md` — live/production verification strategy

The Gate Function

COMPLETION → TRUTH → PROOF

1. **Completion:** Did the agent finish its task? (contract says PASS) 2. **Truth:** Is the work actually correct? (independent verification) 3. **Proof:** Can you prove it with evidence? (exit codes, test output, screenshots)

A PASS without proof is a claim. When a result surprises you in either direction — a pass that came too easily, or a failure you cannot explain — suspect the observation method before the system: confirm the check runs the right command against the current build and exercises the real code path before theorizing about the code.

<!-- Authoring rule (maintenance, not runtime): every gate in this skill encodes an observed failure mode; understand what a gate prevents before removing it. -->

Self-Critique Gate (BEFORE Verification Commands)

Before running any test, audit your own work:

**Code Quality:**

• [ ] No TODO/FIXME/stubs in changed files
• [ ] No commented-out code
• [ ] No debug logging left in
• [ ] Error handling covers the failure modes the change introduces
• [ ] Types are correct (not `as any` or `as unknown`)

**Implementation Completeness:**

• [ ] Every acceptance criterion from the plan has corresponding code
• [ ] Every named scenario has a test
• [ ] Every error path in the plan has handling
• [ ] No silent failures (empty catches, discarded errors)
• [ ] Build succeeds, type-check passes

**Self-Critique Verdict:** If any check fails, fix it BEFORE running verification. Don't run tests you know will fail on code quality issues — a known-failing run produces noise evidence that then has to be explained away.

Validation Levels

| Level | What | Exit Code | When | | ------- | ------ | ----------- | ------ | | **Deterministic** | Automated test | 0/1 | Always — the default | | **Probabilistic** | Test with known flake rate | 0/1 (retry policy) | When deterministic impossible (timing, network) | | **Manual** | Human verifies with checklist | n/a | When automation not worth the cost | | **Live** | Production-like environment | 0/1 | When plan requires live proof |

Every verification must state its validation level — unlabeled evidence lets weak proof masquerade as strong. If manual, state the checklist. Manual evidence is a validation level for verification, never a substitute for a TDD RED or GREEN exit code: with no test runner and no scripted check, require a runner or block. If deterministic, state the command + exit code. If live, state the harness command.

Production-Like Live Proof

When the plan includes `### Live Verification Strategy` or a harness manifest:

• Run `python3 "${CLAUDE_PLUGIN_ROOT}/tools/live_harness_runner.py" --manifest <path> --mode proof`
• If stress required: also run `--mode stress`
• Do NOT silently substitute replay fixtures or unit tests for required live proof

**Flaky test handling:** re-run once — one retry separates environment blips from real flake; more retries launder genuine failures. Pass on re-run → mark PASS with `flaky: true`. Fail both → FAIL. Never convert flaky pass into unconditional confidence.

Evidence Array Protocol (MANDATORY for PASS)

EVIDENCE:
  scenarios:
    - "[name] | Given [state] | When [action] | [command] → exit [code] | expected=[expected] | actual=[actual]"
  regressions:
    - "[test] → exit [code]: [result]"
  edge_cases:
    - "[case]: [command] → exit [code]: [result]"

Every scenario needs non-empty Expected and Actual. Every scenario maps to exactly one EVIDENCE entry. SCENARIOS_PASSED must equal EVIDENCE.scenarios with exit 0 + Result=PASS. Run each check against the revision being verified; if the code changes after a check, re-run the affected scenarios before citing them. Record the tested identity with the evidence: the commit SHA, plus a patch or digest when the tree is dirty; evidence that does not identify the revision it was produced from is unverified, not PASS.

Goal-Backward Lens

Walk backward from the goal to verify it was achieved:

1. **What was the goal?** (re-read the plan's exit criteria) 2. **What would prove it?** (name the specific evidence) 3. **Do I have that evidence?** (run the verification) 4. **Does the evidence actually prove the goal?** (not "tests pass" but "the tests test the right thing")

Common Failures

| Failure | What happens | Fix | | --------- | ------------- | ----- | | **False green** | Test passes without exercising the real code path | Test Honesty Gates (see integration-verifier) | | **Tautological check** | Expected value recomputed the way the code computes it, e.g. `expect(add(a, b)).toBe(a + b)` — passes by construction | Expected values come from an independent source of truth: a known-good literal, a worked example, the spec | | **Scope skip** | "All tests pass" but untested scenarios exist | Goal-backward lens: name every scenario, verify each | | **Stale evidence** | "Tests pass" but you didn't run them this session | Re-run. Evidence must be from THIS session. | | **Claim without proof** | "It works" with no command/exit code | Evidence array is mandatory for PASS | | **Pointer-as-proof** | Path cited as PASS without re-opening it | Re-open the artifact; if it does not show the claim, downgrade or re-run. Never cite it as PASS | | **Untested claim** | A scenario or claim the return never addresses | Mark it untested