/test-smell-detection

name: test-smell-detection
description: >
  Audits existing tests in any language using formal, research-backed test
  smell names and the testsmells.org 19-smell academic taxonomy. Use when the
  caller asks for an academic or citable test-smell review, named smell
  categories, or a formal severity-ranked smell assessment. Covers Assertion
  Roulette, Conditional Test Logic, Mystery Guest, Eager Test, Sleepy Test,
  Unknown Test, Sensitive Equality, and the rest of the catalog across .NET,
  Python, JavaScript/TypeScript, Java, Go, Ruby, Rust, Swift, Kotlin,
  PowerShell, and C++. DO NOT USE FOR a quick pragmatic test review (use
  test-anti-patterns), writing or running tests, framework migration, coverage,
  or assertion-diversity metrics.
license: MIT

Test Smell Detection

Audit test code with the academic taxonomy, code evidence, calibrated framework idioms, and fixes native to the codebase.

Scope

• Audit only staged or named tests. Search the current workspace before asking

for code; never claim a file is missing until that search finds no relevant test.

• Read production code only when it changes a verdict.
• For unfamiliar framework APIs, read the matching language file relative to

the caller-provided or runtime-listed `test-analysis-extensions` catalog, not a skill invocation. If absent, use the framework and rules here and report the missing reference; do not search installation directories.

• Read [the complete catalog](references/test-smell-catalog.md) when the caller

requests all 19 smells, asks for citations, or the code may contain a smell outside the high-signal set below. Do not load it for a narrow question that this file answers.

Audit Workflow

1. Search for and read the staged tests; detect language, framework, boundaries, and integration markers. This is the first action even when no path is named. 2. Read the tests and only verdict-changing production context. 3. For each candidate, verify the executed check, choose the formal category, calibrate, then assign severity. Report proven non-catalog test-validity defects separately; do not relabel them as smells. 4. Rank confirmed findings by risk of false confidence or flakiness, then by maintenance cost. 5. Give a framework-correct replacement for each actionable finding. Never use .NET terminology or APIs in another ecosystem.

High-Signal Decisions

| Evidence | Academic finding | Do | Never | |---|---|---|---| | Assertion behavior changes behind `if`, `switch`, or branching loops | Conditional Test Logic | Split cases or parameterize them | Flag table-driven or parametrized tests merely because a runner loop exists | | A test relies on an undeclared file, network service, environment value, or database | Mystery Guest or Resource Optimism | Make the dependency explicit and hermetic; distinguish the two using the full catalog | Condemn an integration test merely for exercising its declared real resource | | Fixed wall-clock sleep waits for an outcome | Sleepy Test | Await or poll the condition with a timeout | Downgrade it only because the test is an integration test | | Executable test has no assertion, expected-exception marker, or mock verification | Unknown Test | Assert the observable outcome | Call an empty body Unknown Test; the formal name is Empty Test | | Async assertion/coroutine is created but not awaited or returned | Critical non-catalog false-pass defect | Report it separately and show the required `await`/`return` | Force it into Unknown Test; the assertion statement exists | | One test exercises many unrelated production behaviors | Eager Test | Separate behavior-focused tests | Flag a deliberate end-to-end workflow without considering its scope | | Expected numeric literal has no local meaning | Magic Number Test | Name the domain value or derive it from setup | Flag `count == 3` immediately after adding three items | | Assertion depends on `ToString`, `repr`, `description`, or display formatting that is not the contract | Sensitive Equality | Assert stable fields or use a structural matcher | Flag a test whose explicit contract is the formatted string | | Test manually manages expected exception flow | Exception Handling | Use the framework's exception assertion and check meaningful details | Claim a capture-and-assert test verifies nothing | | Shared setup creates state irrelevant to the tests that receive it | General Fixture | Remove unused state or narrow the fixture; rank cheap state low | Condemn relevant shared setup merely because it is shared | | Test is disabled or skipped | Ignored Test | Report every skip, but rank a tracked, reasoned skip below an unexplained one | Clear a skip because its reason is good, or give both the same urgency |

Calibration Rules

Apply these before assigning a finding:

• Mock-call verifications, snapshots, bare pytest `assert`, Pester

`Should -Invoke`, and expected-exception constructs are assertions.

• A literal or snapshot assertion may expose a coverage gap, but is not Unknown

Test or another smell without separate evidence.

• Count assertion statements. One assertion is never Assertion Roulette;

missing messages alone are not a smell.

• Same-method tests are not Lazy Test when they cover distinct behaviors,

boundaries, or state; require redundant equivalent paths.

• General Fixture requires shared lifecycle state. Repeated local construction

is neither General Fixture nor Test Code Duplication by itself.

• Treat strings returned by the public API as observable contract unless

production context or requirements make them display-only; interpolation alone is not Sensitive Equality.

• Magic Number Test requires an unexplained oracle value. Do not flag ordinary

setup quantities whose role is locally obvious and irrelevant to the asserted behavior.

• Go table-driven subtests, pytest/JUnit/xUnit parameterization, Jest/Vitest

`.each`, RSpec data tables, Pester `-ForEach`, and Catch2