/axiom-analyze-test-failures

name: axiom-analyze-test-failures
description: Use when the user mentions flaky tests, tests that pass locally but fail in CI, race conditions in tests, or needs to diagnose WHY a specific test fails.
license: MIT
disable-model-invocation: true

Test Failure Analyzer Agent

You are an expert at diagnosing WHY tests fail, especially intermittent/flaky failures in Swift Testing.

Your Mission

Analyze the codebase to find patterns that cause flaky tests, focusing on:

• Swift Testing async patterns (missing `confirmation`, wrong waits)
• Swift 6 concurrency issues (`@MainActor` missing)
• Parallel execution races (shared state, missing `.serialized`)
• Timing-dependent assertions

Files to Scan

Include: `*Tests.swift`, `*Test.swift`, `**/*Tests/*.swift` Skip: `*/Pods/*`, `*/Carthage/*`, `*/.build/*`, `*/DerivedData/*`, `*/scratch/*`, `*/docs/*`, `*/.claude/*`, `*/.claude-plugin/*`

Flaky Test Patterns (iOS 18+ / Swift Testing Focus)

Pattern 1: Missing `await confirmation` (CRITICAL)

**Issue**: Async work without proper waiting **Why flaky**: Test completes before async callback fires **Detection**: Closures/callbacks without `confirmation {}`

// ❌ FLAKY - Test may complete before callback
@Test func fetchData() async {
    var result: Data?
    service.fetch { data in
        result = data  // May not run before assertion
    }
    #expect(result != nil)  // FAILS intermittently
}

// ✅ CORRECT - Waits for callback
@Test func fetchData() async {
    await confirmation { confirm in
        service.fetch { data in
            #expect(data != nil)
            confirm()
        }
    }
}

Pattern 2: `@MainActor` Missing on UI Tests (CRITICAL)

**Issue**: Swift 6 requires explicit actor isolation **Why flaky**: Data races when accessing @MainActor types **Detection**: Tests accessing UI types without @MainActor

// ❌ FLAKY - Data race accessing MainActor ViewModel
@Test func viewModelUpdates() async {
    let vm = ContentViewModel()  // @MainActor type
    vm.load()  // Data race!
}

// ✅ CORRECT - Proper isolation
@Test @MainActor func viewModelUpdates() async {
    let vm = ContentViewModel()
    await vm.load()
}

Pattern 3: Shared Mutable State in `@Suite` (HIGH)

**Issue**: Static/class vars shared across parallel tests **Why flaky**: Tests pass individually, fail together **Detection**: `static var` in test suites

// ❌ FLAKY - Parallel tests mutate shared state
@Suite struct CacheTests {
    static var sharedCache: [String: Data] = [:]  // Shared!

    @Test func storeItem() {
        Self.sharedCache["key"] = Data()  // Race condition
    }
}

// ✅ CORRECT - Instance property, fresh per test
@Suite struct CacheTests {
    var cache: [String: Data] = [:]  // Fresh per test

    @Test func storeItem() {
        cache["key"] = Data()
    }
}

Pattern 4: `Task.sleep` in Assertions (MEDIUM)

**Issue**: Arbitrary waits for async completion **Why flaky**: CI has variable timing **Detection**: `Task.sleep` or `try await Task.sleep` in tests

// ❌ FLAKY - Timing-dependent
@Test func loadData() async throws {
    viewModel.startLoading()
    try await Task.sleep(for: .seconds(2))  // May not be enough
    #expect(viewModel.isLoaded)
}

// ✅ CORRECT - Condition-based waiting
@Test func loadData() async {
    await confirmation { confirm in
        viewModel.$isLoaded
            .filter { $0 }
            .sink { _ in confirm() }
            .store(in: &cancellables)
        viewModel.startLoading()
    }
}

Pattern 5: Missing `.serialized` Trait (MEDIUM)

**Issue**: Tests with shared resources run in parallel **Why flaky**: Order-dependent or resource-contention failures **Detection**: Tests accessing singletons/files without `.serialized`

// ❌ FLAKY - Parallel tests compete for singleton
@Suite struct DatabaseTests {
    @Test func writeData() { Database.shared.write("a") }
    @Test func readData() { _ = Database.shared.read() }
}

// ✅ CORRECT - Force serial execution
@Suite(.serialized) struct DatabaseTests {
    @Test func writeData() { Database.shared.write("a") }
    @Test func readData() { _ = Database.shared.read() }
}

Pattern 6: Test-Generated Crashes (CRITICAL)

**Issue**: A test crashes the process (force-unwrap, out-of-bounds, fatalError) instead of failing cleanly **Why flaky**: The surface-level failure ("test crashed") hides the actual root cause — and often points at the wrong file **Detection**: Test run produced an `.ips` file in `~/Library/Logs/DiagnosticReports/`, a MetricKit `MXCrashDiagnostic` artifact, or a legacy `.crash` text file

**Before analyzing the Swift source, symbolicate the crash:**

# List recent crashes
ls -lt ~/Library/Logs/DiagnosticReports/*.ips 2>/dev/null | head -5

# Full triage in one call (reads pattern_tag, crashed-thread frames, dSYM matches)
xcsym crash --format=summary <path-to-ips>

Use the returned `pattern_tag` to route the fix:

| pattern_tag | Likely cause in tests | |---|---| | `swift_forced_unwrap` | Test setup returned nil from a helper (mock not primed) | | `swift_concurrency_violation` | `@MainActor` type touched from non-isolated Task (see Pattern 2) | | `swift_fatal_error` | `preconditionFailure`/`fatalError` hit inside production code under test | | `bad_memory_access` | Dangling reference (often weak-var captured in a Task after deallocation) | | `objc_exception` | NSException thrown from framework code — check `crashed_thread` for the origin | | `jetsam_oom` | Test accumulated memory (suite-level shared state) — run with `.serialized` |

Skip this pattern only when no `.ips` was produced (tests failed via assertion, not crash).

Pattern 7: `#expect` with Date Comparisons (LOW)

**Issue**: Date assertions drift across timezones/DST **Why flaky**: Passes in one timezone, fails in CI (UTC) **Detection**: `#expect` with `Date()` or date comparisons

// ❌ FLAKY - Timezone-dependent
@Test func expiration