/flaky-fix

allowed-tools: Task, Read, Write, Edit, MultiEdit, Bash(fd:*), Bash(rg:*), Bash(bat:*), Bash(jq:*), Bash(gdate:*), Bash(git:*), Bash(gh:*), Bash(docker:*), Bash(deno:*), Bash(cargo:*), Bash(go:*), Bash(mvn:*), Bash(gradle:*), Bash(python:*), Bash(pytest:*), Bash(jest:*), Bash(wc:*), Bash(head:*), Bash(tail:*), Bash(sort:*), Bash(uniq:*)
name: "Flaky Fix"
description: "Intelligent flaky test detection and stabilization system with automated pattern recognition and framework-specific fixes"
author: "wcygan"
tags: ["test","fix"]
version: "1.0.0"
created_at: "2025-07-14T00:00:00Z"
updated_at: "2025-07-14T00:00:00Z"

Context

• Session ID: !`gdate +%s%N 2>/dev/null || date +%s%N 2>/dev/null || echo "$(date +%s)$(jot -r 1 100000 999999 2>/dev/null || shuf -i 100000-999999 -n 1 2>/dev/null || echo $RANDOM$RANDOM)"`
• Target test: $ARGUMENTS
• Current directory: !`pwd`
• Test framework detected: !`if [ -f "deno.json" ]; then echo "deno"; elif [ -f "Cargo.toml" ]; then echo "rust"; elif [ -f "go.mod" ]; then echo "go"; elif [ -f "package.json" ]; then echo "node"; elif [ -f "pyproject.toml" ] || [ -f "requirements.txt" ]; then echo "python"; else echo "unknown"; fi`
• Test files found: !`fd "(test|spec)" --type f | head -10 || echo "No test files detected"`
• CI history available: !`if [ -d ".github/workflows" ]; then echo "GitHub Actions detected"; elif [ -f ".gitlab-ci.yml" ]; then echo "GitLab CI detected"; else echo "No CI detected"; fi`
• Git repository: !`git status --porcelain | wc -l | tr -d ' '` uncommitted changes
• Recent test failures: !`git log --since="7 days ago" --oneline --grep="test.*fail\|fix.*test\|flaky" | wc -l | tr -d ' '` commits

Your Task

STEP 1: Initialize comprehensive flaky test analysis session

• CREATE session state file: `/tmp/flaky-test-session-$SESSION_ID.json`
• ANALYZE project structure and testing framework from Context section
• DETERMINE scope: specific test vs. project-wide analysis based on $ARGUMENTS
• VALIDATE required tools and testing environment

# Initialize flaky test analysis session
echo '{
  "sessionId": "'$SESSION_ID'",
  "targetTest": "'$ARGUMENTS'",
  "testFramework": "auto-detect",
  "analysisScope": "determine-from-args",
  "flakyPatterns": [],
  "stabilizationActions": [],
  "monitoringSetup": false
}' > /tmp/flaky-test-session-$SESSION_ID.json

STEP 2: Parallel comprehensive analysis using sub-agent coordination

IF project_complexity == "large" OR scope == "project-wide":

LAUNCH parallel sub-agents for comprehensive flaky test investigation:

• **Agent 1: Test History Mining**: Analyze git history and CI failures for flake patterns
• Focus: Git commit analysis, CI failure extraction, test modification patterns
• Tools: git log analysis, gh CLI for GitHub Actions, pattern extraction with rg
• Output: Historical flake data and frequency analysis

• **Agent 2: Framework-Specific Detection**: Analyze test code for flaky patterns by framework
• Focus: Timeout issues, race conditions, randomness, async patterns
• Tools: rg for pattern matching, fd for test file discovery, framework-specific analysis
• Output: Code-level flaky pattern identification and severity assessment

• **Agent 3: CI/CD Integration Analysis**: Extract test results and failure patterns from CI systems
• Focus: GitHub Actions logs, test result artifacts, failure frequency
• Tools: gh CLI, jq for JSON processing, log analysis
• Output: CI-based flake statistics and automation insights

• **Agent 4: Test Environment Assessment**: Analyze test configuration and environment stability
• Focus: Test setup, teardown, isolation, external dependencies
• Tools: Configuration file analysis, dependency detection
• Output: Environment stability recommendations

ELSE:

EXECUTE focused single-test analysis workflow:

Context Intelligence

STEP 3: Intelligent test framework detection and adaptation

TRY:

**Framework-Adaptive Analysis Pattern:**

# Enhanced framework detection with tool validation
detect_test_framework() {
  local session_id="$SESSION_ID"
  
  if [ -f "deno.json" ]; then
    FRAMEWORK="deno"
    TEST_COMMAND="deno test"
    TEST_FILES=$(fd "(test|spec)" --extension ts --extension js | head -20)
    STABILITY_TOOLS="deno_test_utils"
  elif [ -f "Cargo.toml" ]; then
    FRAMEWORK="rust"
    TEST_COMMAND="cargo test"
    TEST_FILES=$(fd "_test\.rs$|tests/" --type f | head -20)
    STABILITY_TOOLS="tokio_test"
  elif [ -f "go.mod" ]; then
    FRAMEWORK="go"
    TEST_COMMAND="go test"
    TEST_FILES=$(fd "_test\.go$" | head -20)
    STABILITY_TOOLS="testify"
  elif [ -f "package.json" ]; then
    FRAMEWORK="node"
    TEST_COMMAND="npm test"
    TEST_FILES=$(fd "(test|spec)\.(js|ts|jsx|tsx)$" | head -20)
    STABILITY_TOOLS="jest_retry"
  elif [ -f "pyproject.toml" ] || [ -f "requirements.txt" ]; then
    FRAMEWORK="python"
    TEST_COMMAND="pytest"
    TEST_FILES=$(fd "test_.*\.py$|.*_test\.py$" | head -20)
    STABILITY_TOOLS="pytest_rerunfailures"
  else
    echo "❌ Unknown testing framework - manual analysis required"
    exit 1
  fi
  
  # Update session state with framework details
  jq --arg fw "$FRAMEWORK" --arg cmd "$TEST_COMMAND" --arg tools "$STABILITY_TOOLS" '
    .testFramework = $fw | 
    .testCommand = $cmd | 
    .stabilityTools = $tools
  ' /tmp/flaky-test-session-$session_id.json > /tmp/flaky-test-session-$session_id.tmp && 
  mv /tmp/flaky-test-session-$session_id.tmp /tmp/flaky-test-session-$session_id.json
}

detect_test_framework

STEP 4: Advanced CI/CD integration with intelligent failure pattern extraction

**Enhanced CI History Analysis:**

# Advanced GitHub Actions analysis with error categorization
analyze_github_ci() {
  local session_id="$SESSION_ID"
  
  if [ -d ".github/workflows" ]; then
    echo "🔍 Analyzing GitHub Actions test history..."
    
    # Extract comprehensive CI data
    gh run list --limit 100 --json status,conclusion,createdAt,workflowName,databaseId > /tm