/assertion-quality

name: assertion-quality
description: "Analyze assertion quality, depth, variety, and false confidence in existing tests. ALWAYS USE when asked about weak, shallow, trivial, always-true, self-referential, assertion-free, presence/truthiness-only, or insufficiently diverse assertions, including MSTest, Jest, pytest, and Go. DO NOT USE for direct fixes: writing-mstest-tests owns supplied MSTest assertions; code-testing owns new cases. Use test-gap-analysis when asked whether tests would catch a production change, and test-anti-patterns for general severity-ranked audits."
license: MIT

Assertion Diversity Analysis

Analyze test code in any supported language to measure how varied and meaningful the assertions are. Produce a metrics report that reveals whether tests verify different facets of correctness — not just "output equals X" but also structure, exceptions, state transitions, side effects, and invariants.

> **Language-specific guidance**: Read the caller-provided or runtime-listed > `test-analysis-extensions` catalog, then its matching language file > (`dotnet.md`, `python.md`, `typescript.md`, `go.md`, etc.). > `test-analysis-extensions` is reference-only; do not invoke it as a skill. > If the bundle is absent, use the pinned framework and this skill's rules, > report the missing reference, and do not search installation directories.

Why Assertion Diversity Matters

Low assertion diversity signals shallow testing. Tests may pass while bugs hide in unasserted logic. Common symptoms:

| Problem | Symptom | Consequence | |---------|---------|-------------| | Trivial assertions | Test contains only `Assert.IsNotNull(result)` / `assert result is not None` / `expect(x).toBeDefined()` | Test passes but doesn't verify correctness | | Single-value obsession | Always check one field or return value | Bugs in unasserted logic slip through | | No negative assertions | Never check what shouldn't happen | Regressions sneak in through false positives | | No state checks | Don't verify object state changes | Missed side-effects or lifecycle issues | | No structural checks | Only assert top-level value | Bugs in nested objects go unnoticed | | Assertion-free tests | Tests that call but don't verify | Code coverage lies; false security |

When to Use

• User asks to evaluate assertion quality or depth
• User asks "are my tests actually testing anything meaningful?"
• User wants to know if test assertions are too shallow or trivial
• User asks for assertion coverage metrics or diversity analysis
• User suspects tests give false confidence despite passing
• The `test-engineer` agent (or any test-generation workflow) calls this skill as a pre-completion self-review step on freshly generated tests, before declaring the run finished

When Not to Use

• User wants to write new tests (use `code-testing` for any language, or `writing-mstest-tests` for MSTest specifically)
• User wants to detect anti-patterns beyond assertions (use `test-anti-patterns`)
• User wants to fix or rewrite assertions (help them directly)
• User asks about code coverage percentages (out of scope — this analyzes assertion quality, not line coverage)

Inputs

| Input | Required | Description | |-------|----------|-------------| | Test code | Yes | One or more test files or a test project directory to analyze | | Production code | No | The code under test, to evaluate whether assertions cover the important behaviors |

Workflow

Step 1: Detect language and load extension

Identify the language and test framework. Read the corresponding file in `extensions/` relative to the supplied `test-analysis-extensions` catalog. Use that catalog for other language filenames. Check only that known directory when necessary; an unavailable reference changes the evidence limit, not the workspace root or the requested review.

Step 2: Gather the test code

Read all test files the user provides. If the user points to a directory or project, scan for all test files using the markers in the language extension file (e.g., `[TestMethod]` for MSTest, `def test_*` for pytest, `it()` / `test()` for Jest, `func TestXxx` for Go).

Step 3: Classify every assertion

For each test method, identify all assertions and classify them into these language-neutral categories:

| Category | What it verifies | Examples across languages | |----------|------------------|----------------------------| | **Equality** | Return value matches expected | `Assert.AreEqual` (MSTest), `Assert.Equal` (xUnit), `assert x == y` (pytest), `expect(x).toBe(y)` (Jest), `assertEquals` (JUnit), `if got != want { t.Error... }` / `assert.Equal(t, want, got)` (Go), `x shouldBe y` (Kotest), `Should -Be` (Pester), `EXPECT_EQ` (GoogleTest) | | **Boolean** | Condition holds | `Assert.IsTrue`, `assert flag` (Python), `expect(x).toBeTruthy()` (Jest), `assertTrue` (JUnit), `assert.True(t, ok)` (testify), `x.shouldBeTrue()` (Kotest), `Should -BeTrue` (Pester), `EXPECT_TRUE` | | **Null / None / Nil** | Presence/absence of value | `Assert.IsNull` (.NET), `assert x is None` (pytest), `expect(x).toBeNull()` (Jest), `assertNull` (JUnit), `assert.Nil(t, v)` (testify), `XCTAssertNil` (XCTest), `Should -BeNullOrEmpty` (Pester) | | **Exception / Error** | Error handling behavior | `Assert.Throws<T>()`, `pytest.raises(E)`, `expect(fn).toThrow(E)`, `assertThrows<E>`, `assert.Error(t, err)` / `assert.ErrorIs`, `#[should_panic]` (Rust), `XCTAssertThrowsError`, `Should -Throw`, `EXPECT_THROW` | | **Type checks** | Runtime type correctness | `Assert.IsInstanceOfType`, `assert isinstance(x, T)`, `expect(x).toBeInstanceOf(T)`, `assertInstanceOf`, `assert.IsType(t, T{}, v)`, `assert!(matches!(value, Pattern))` (Rust), `Should -BeOfType` | | **String** | Text content and format | `StringAssert.Contains`, `assert sub in s`, `expect(s).toMatch(/x/)`, `assertTrue(s.contains(...))`, `assert.Contains(t, s, sub)`, `s shouldContain sub`, `Should -Match`, `EXPECT_THAT(s, HasSubstr(...))` | | **C