/ln-55-test-suite-auditor

name: ln-55-test-suite-auditor
description: "Audits existing tests for risk coverage, reliable oracles and maintenance value; does not edit tests."

Test Suite Auditor

**Goal:** Audit the test portfolio as a read-only lifecycle and confidence system. Determine which important failures it detects, which evidence is untrustworthy or obsolete, and which additions, changes, consolidations, retirements, or explicit omissions produce the smallest sustainable portfolio.

**Execution contract:** The checklist defines completion. Track each item internally as `PENDING`, `PROVEN` with evidence, `CLEARED` with evidence its condition is absent, or `UNPROVEN` with a gap; reading, delegation, or tool failure is not proof. Reconcile after each section. Before returning, resolve all `PENDING`, count only `PROVEN` and `CLEARED`, and apply verdict and approval rules to every gap. Preserve intent, scope, and existing authorization. Continue authorized work; ask only for consequential unresolved choices or required external approval. Scale depth to material risk without skipping checks. Preserve dependency and safety order; otherwise choose an appropriate verification method. Accept equivalent user or repository evidence; no other skill, named artifact, or complete lifecycle is required. Preserve source requirement and decision IDs. Bind reused evidence to relevant source versions, dirty changes, configuration, and environment; invalidate only affected claims. On continuation, reconcile task, authorization, current state, and unresolved evidence. For long work, return a compact continuation record or update an already authorized artifact; read-only skills do not persist it. Distinguish artifact readiness, verified behavior, and external-action authority. Prepare authorized work before required approval. If blocked by an instruction, cite its exact source and unresolved boundary; do not invent approval gates from caution.

Tool Routing

| Need | Preferred tool | Use it when | Fallback | |---|---|---|---| | Source and test inventory | Native file listing, search, manifests, and test configuration | Mapping domains, test types, runners, fixtures, and generated areas | Repository tree plus known test entrypoints | | Test-to-code relationships | Language server or host-native code intelligence | Mapping units, callers, implementations, routes, and test targets | Naming and path search verified by direct reads | | Execution and trust | Repository-defined test commands through the shell | Establishing pass/fail state, timing, order dependence, or reproducibility | Inspect CI results and configuration; mark execution unavailable | | Coverage and missed behavior | Existing coverage tools and reports | Coverage data is configured and comparable to source scope | Static behavior-to-test mapping; never invent percentages | | Flake and isolation evidence | Repeated, shuffled, parallel, or seed-controlled runs supported by the repository | A test is suspected of order, time, randomness, or shared-state dependence | History, CI logs, and code-path evidence | | Assertion strength | Test reads, failure output, and configured mutation testing | Determining whether tests fail for meaningful behavioral defects | Counterfactual reasoning tied to specific assertions | | Framework semantics | Official test-runner or framework documentation | A finding depends on lifecycle, fixtures, retries, isolation, or mocking behavior | Primary-source web research; otherwise mark `UNVERIFIED` |

Run only safe test and diagnostic commands. Do not rewrite snapshots, update golden files, regenerate fixtures, or accept changed output during the audit.

Evidence Rules

• Coverage indicates execution, not proof. Require an assertion or observable oracle for important behavior.
• A slow test is not low-value when it uniquely protects a critical journey; a fast test is not high-value when it proves framework behavior.
• A flaky failure must be separated from an intermittently failing product dependency or genuinely nondeterministic requirement.
• Deletion recommendations require proof that the test basis is obsolete or that other evidence covers every still-required behavior and failure mode with equal or better trust.
• Merge recommendations require demonstrated duplicate or fragmented coverage and must preserve distinct business and failure scenarios, oracle strength, and failure localization; a larger test is not inherently better.
• Known regression guards and the only proof of a rare critical edge case are not deletion candidates merely because a numeric value heuristic is low.
• A real dependency is not inherently a test defect. Judge whether its version, state, ownership, reset, availability, and failure evidence make the result reproducible.
• Keep portfolio action (`KEEP`, `ADD`, `UPDATE`, `MERGE`, `DELETE`, `NO_TEST`) separate from execution status (`PASS`, `FAIL`, `BLOCKED`, `UNPROVEN`, `QUARANTINED`). `NO_TEST` requires existing proof, another control, or explicit residual-risk acceptance.
• Do not require a new registry by default. Prefer traceability derivable from repository-native test paths, behavioral names, tags, requirements, CI configuration, and review evidence.
• External testing guidance becomes actionable only when it explains a concrete weakness in this suite.

Checklist

1. Map the Portfolio and Baseline

• [ ] Detect test runners, configurations, commands, directories, fixtures, factories, snapshots, golden files, manual scripts, coverage, and mutation tooling.
• [ ] Map source domains and critical entrypoints to unit, integration, contract, end-to-end, and manual test surfaces.
• [ ] Trace requirements, product risks, incidents, public contracts, and changed behavior to tests and results where evidence exists; identify tests with no current test basis and basis elements with no credible proof.
• [ ] Read repository instructions and CI configuration to identify required suites, environment assumpti