/test-suite-analysis

name: test-suite-analysis
description: Layer 1 skill for extracting behavioral intelligence from test suites. Framework detection, test code reading strategy, test execution strategy, behavioral claim extraction with Given/When/Then mapping, e2e vs unit value classification. Loaded by the analyzer agent during Layer 1.

Test Suite Analysis Methodology

Extract behavioral intelligence from test suites. Tests are executable specifications -- they encode what the system MUST do in a form that can be verified. A passing test is a confirmed behavioral contract.

When to Use This Mode

Test suite analysis activates when:

• The target repository contains test files
• The discovery inventory identifies test files in the project
• Other modes discover test directories during analysis

This mode runs independently of all other intelligence sources. All output is **RAW** (test code references internal implementation details).

Why Tests Are High-Value Intelligence

Tests are the only source type that is simultaneously:

• **Behavioral** -- they describe what the system does, not how it's built
• **Executable** -- they can be run to confirm the behavior still holds
• **Specific** -- they provide exact inputs, expected outputs, and edge cases
• **Maintained** -- failing tests get fixed, so they track current behavior

A single end-to-end test is worth more than a page of documentation because the test is verified by CI on every commit.

Framework Detection

Identify the test framework(s) in use before analyzing test code. Different frameworks use different assertion styles, test organization, and execution models.

| Framework | Language | Detection Signals | |-----------|----------|-------------------| | Jest | JavaScript/TypeScript | `jest.config.*`, `describe(` / `it(` / `expect(` in `__tests__/` or `*.test.*`, `@jest/globals` imports | | Playwright | JavaScript/TypeScript | `playwright.config.*`, `@playwright/test` imports, `page.goto(` / `page.click(` | | Cypress | JavaScript/TypeScript | `cypress.config.*`, `cypress/` directory, `cy.visit(` / `cy.get(` | | pytest | Python | `conftest.py`, `pytest.ini` / `pyproject.toml` with `[tool.pytest]`, files named `test_*.py` / `*_test.py`, `assert` statements | | Go testing | Go | `*_test.go` files, `testing.T` / `testing.B` parameters, `go test` in CI config | | RSpec | Ruby | `.rspec`, `spec/` directory, `spec_helper.rb`, `describe` / `it` / `expect` blocks | | JUnit | Java/Kotlin | `@Test` annotations, `src/test/` directory, `assertEquals` / `assertThat` calls | | XCTest | Swift/Objective-C | `XCTestCase` subclasses, `func test*()` methods, `XCTAssert*` calls | | Catch2 | C++ | `#include <catch2/catch.hpp>`, `TEST_CASE(` / `SECTION(` / `REQUIRE(` macros |

Detection Strategy

# Check for test configuration files
ls -la jest.config.* playwright.config.* cypress.config.* .rspec pytest.ini 2>/dev/null

# Check pyproject.toml for pytest config
grep -l '\[tool\.pytest' pyproject.toml 2>/dev/null

# Find test directories
find . -maxdepth 3 -type d \( -name "__tests__" -o -name "test" -o -name "tests" -o -name "spec" -o -name "cypress" \) 2>/dev/null

# Find test files by naming convention
find . -maxdepth 4 -type f \( -name "*.test.*" -o -name "*.spec.*" -o -name "test_*" -o -name "*_test.*" \) 2>/dev/null | head -50

# Count test files per pattern
echo "Jest/Mocha-style:" && find . -name "*.test.*" -o -name "*.spec.*" 2>/dev/null | wc -l
echo "Python-style:" && find . -name "test_*.py" -o -name "*_test.py" 2>/dev/null | wc -l
echo "Go-style:" && find . -name "*_test.go" 2>/dev/null | wc -l
echo "JUnit-style:" && find . -path "*/src/test/*" -name "*.java" 2>/dev/null | wc -l

Write detection results to `workspace/raw/test-evidence/test-inventory.md`.

Strategy 1: Read Test Code

Read test files directly and extract behavioral claims. This strategy always works -- it requires no working environment, no dependencies, and no execution.

1.1 Test File Inventory

# Build complete inventory of test files with metadata
find . -type f \( -name "*.test.*" -o -name "*.spec.*" -o -name "test_*" -o -name "*_test.*" -o -name "*_test.go" \) 2>/dev/null | while read f; do
  lines=$(wc -l < "$f")
  echo "$lines $f"
done | sort -rn

1.2 Assertion Extraction

For each test file, extract the assertions -- these are the behavioral contracts:

# Jest/Mocha assertions
grep -n "expect\|assert\|should\|toBe\|toEqual\|toContain\|toThrow\|toHaveBeenCalled" "$TEST_FILE"

# pytest assertions
grep -n "assert \|assert_\|assertEqual\|assertRaises\|pytest.raises" "$TEST_FILE"

# Go test assertions
grep -n "t\.Error\|t\.Fatal\|t\.Log\|assert\.\|require\." "$TEST_FILE"

# RSpec assertions
grep -n "expect\|should\|is_expected\|eq(\|include(\|raise_error" "$TEST_FILE"

1.3 Given/When/Then Extraction

Transform test code into behavioral claims using Given/When/Then structure:

For each test case (`it(`, `test(`, `func Test*`, `def test_*`), extract:

• **Given** (setup/preconditions): fixture creation, mock configuration, state initialization
• **When** (action): the function call, API request, or user action being tested
• **Then** (assertions): the expected outcomes encoded in assertions

## Test: "should reject expired tokens"

**Given:** A token with expiry date in the past
**When:** The token is validated via `checkPermissions()`
**Then:**
- Returns false
- Sets error to "TOKEN_EXPIRED"
- Does not call the downstream service

**Source:** `auth.test.ts:45-62`
**Confidence:** confirmed (test assertion is an explicit behavioral contract)

1.4 E2E vs Unit Value Classification

Not all tests carry equal behavioral intelligence value:

| Test Type | Detection Signals | Behavioral Value | |-----------|-------------------|-----------------| | End-to-end (e2e) | Browser automation, HTTP requests to running server, multi-service interaction | **High** -- tests the system as a user experiences it | | Integra