/test-audit

description: Fan out subagents to audit test quality across the repo
argument-hint: [context]
allowed-tools: Bash, Read, Grep, Glob, Task, AskUserQuestion

Audit Tests

You are performing a comprehensive audit of test quality across the repository (or a specified scope). The goal is to identify fake, meaningless, or low-quality tests — particularly those generated by AI that pass trivially without testing real behavior.

**Context (optional):** $ARGUMENTS

Step 1: Discover Tests

Find all test files in the repository (or specified scope):

• **Python**: Look for `test_*.py`, `*_test.py`, files in `tests/` directories
• **JavaScript/TypeScript**: Look for `*.test.js`, `*.spec.ts`, files in `__tests__/` directories
• **Other**: Detect the project's test framework from config files (pytest.ini, jest.config.js, etc.)

If context was provided ($ARGUMENTS): if it looks like a file path or glob pattern, limit discovery to that directory or pattern; if it reads as prose, keep discovery repo-wide but note it as quality-focus guidance for Step 3 agents.

Step 2: Categorize

Group test files into logical categories by:

• Module or feature area they test
• Test type (unit, integration, e2e)
• Size (number of test cases)

Present the categories to the user as a summary table (category name, file count, test count).

Step 3: Fan Out Review Agents

For each category, delegate that category's test files to a `general-purpose` subagent for review. If prose context was provided in $ARGUMENTS, include it in each agent's prompt to steer the quality assessment (e.g., "focus on mocking patterns" or "prioritize the auth module"). Each agent should evaluate:

**Quality criteria:**

• **Fake assertions**: Tests that assert `True`, `is not None` on values that can never be None, or other tautological checks
• **Mocked-everything tests**: Tests where every dependency is mocked, so the test only verifies mock wiring rather than real behavior
• **AI slop**: Boilerplate tests that were clearly auto-generated and pass trivially (e.g., "test that function returns without error" with no meaningful assertion)
• **Missing edge cases**: Tests that only cover the happy path for functions with known edge cases
• **Dead tests**: Tests that are skipped, commented out, or unreachable
• **Incorrect tests**: Tests whose assertions don't actually verify the described behavior

Each agent should return findings as a structured list with:

• File path and test name
• Issue type (fake / mocked-everything / slop / missing-edge-cases / dead / incorrect)
• Severity (critical / moderate / low)
• Brief explanation
• Suggested fix or replacement

Launch agents in parallel where possible.

Do not run these subagents in the background. For parallel execution, launch them in a single message instead.

Step 4: Aggregate Results

After all agents complete, aggregate findings:

1. **Summary statistics**: Total tests audited, issues found by type and severity 2. **Critical findings**: Tests that are actively misleading about code correctness 3. **Moderate findings**: Tests with low value that should be improved 4. **Low findings**: Minor quality issues or style concerns

Step 5: Present Report

Present the full audit report to the user. For critical and moderate findings, include:

• The specific test and what's wrong with it
• A concrete suggestion for how to fix it

Use `AskUserQuestion` to ask the user how they want to proceed:

• **Fix critical issues now**: "Address the most severe test quality problems in this session"
• **Create a GitHub issue**: "Track the cleanup work as a new issue for later"
• **Review specific findings**: "Examine individual findings in detail before deciding"

If the user selects "Create a GitHub issue", draft an issue summarizing the audit findings and offer to create it with `/mach10:issue-create`. If the user selects "Review specific findings", walk through the findings they want to explore, then ask again how to proceed.