/testing-quality-standards

name: testing-quality-standards
description: Defines testing quality metrics, coverage thresholds, and anti-patterns. Use when establishing test gates or validating a test suite's coverage targets.
globs: "**/test_*.py"
alwaysApply: false
category: infrastructure
tags:
- testing
- quality
- standards
- metrics
dependencies: []
estimated_tokens: 400
provides:
  patterns:
  - coverage-thresholds
  - quality-metrics
  - anti-patterns
  - content-assertion-levels
modules:
- modules/anti-patterns.md
- modules/best-practices.md
- modules/content-assertion-levels.md
model_hint: standard

Testing Quality Standards

Shared quality standards and metrics for testing across all plugins in the Claude Night Market ecosystem.

When To Use

• Establishing test quality gates and coverage targets
• Validating test suite against quality standards

When NOT To Use

• Exploratory testing or spike work
• Projects with established quality gates that meet requirements

Table of Contents

1. [Coverage Thresholds](#coverage-thresholds) 2. [Quality Metrics](#quality-metrics) 3. [Detailed Topics](#detailed-topics)

Coverage Thresholds

| Level | Coverage | Use Case | |-------|----------|----------| | Minimum | 60% | Legacy code | | Standard | 80% | Normal development | | High | 90% | Critical systems | | detailed | 95%+ | Safety-critical |

Quality Metrics

Structure

• [ ] Clear test organization
• [ ] Meaningful test names
• [ ] Proper setup/teardown
• [ ] Isolated test cases

Coverage

• [ ] Critical paths covered
• [ ] Edge cases tested
• [ ] Error conditions handled
• [ ] Integration points verified

Maintainability

• [ ] DRY test code
• [ ] Reusable fixtures
• [ ] Clear assertions
• [ ] Minimal mocking

Reliability

• [ ] No flaky tests
• [ ] Deterministic execution
• [ ] No order dependencies
• [ ] Fast feedback loop

Detailed Topics

For implementation patterns and examples:

• **[Anti-Patterns](modules/anti-patterns.md)** - Common testing mistakes with before/after examples
• **[Best Practices](modules/best-practices.md)** - Core testing principles and exit criteria
• **[Content Assertion Levels](modules/content-assertion-levels.md)** - L1/L2/L3 taxonomy for testing LLM-interpreted markdown files

Integration with Plugin Testing

This skill provides foundational standards referenced by:

• `pensive:test-review` - Uses coverage thresholds and quality metrics
• `parseltongue:python-testing` - Uses anti-patterns and best practices
• `sanctum:test-*` - Uses quality checklist and content assertion levels for test validation
• `imbue:proof-of-work` - Uses content assertion levels to enforce Iron Law on execution markdown

Reference in your skill's frontmatter:

dependencies: [leyline:testing-quality-standards]

**Verification:** Run `pytest -v` to verify tests pass.

Troubleshooting

Common Issues

**Tests not discovered** Ensure test files match pattern `test_*.py` or `*_test.py`. Run `pytest --collect-only` to verify.

**Import errors** Check that the module being tested is in `PYTHONPATH` or install with `pip install -e .`

**Async tests failing** Install pytest-asyncio and decorate test functions with `@pytest.mark.asyncio`

Exit Criteria

• [ ] Coverage threshold met for the project tier: 60% minimum for

legacy code, 80% for normal development, 90% for critical systems, 95%+ for safety-critical; measured with `pytest --cov` and threshold enforced in `pyproject.toml`

• [ ] All four quality metric checklists pass: Structure (clear

organization, meaningful names, setup/teardown, isolation), Coverage (critical paths, edge cases, error conditions, integration points), Maintainability (DRY fixtures, clear assertions, minimal mocking), Reliability (no flaky tests, deterministic execution, no order dependencies)

• [ ] Test files match discovery pattern `test_*.py` or `*_test.py`

confirmed by `pytest --collect-only` returning no errors

• [ ] No snapshot tests present on non-deterministic output (hash

maps, timestamps, UUIDs); any found flagged as anti-patterns per `modules/anti-patterns.md`