/aqe-benchmark

name: aqe-benchmark
description: Run performance benchmarks and compare against baselines

AQE Performance Benchmarking

Run performance benchmarks and compare against baselines.

Usage

aqe benchmark <target> [options]
# or
/aqe-benchmark <target> [options]

Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | `target` | string | **required** | Benchmark target: api, database, function, system | | `--baseline` | path | - | Baseline file for comparison | | `--iterations` | number | `1000` | Number of benchmark iterations | | `--warmup` | number | `100` | Warmup iterations | | `--concurrency` | number | `1` | Concurrent requests | | `--duration` | number | - | Duration in seconds (alternative to iterations) | | `--method` | string | `GET` | HTTP method for API benchmarks | | `--payload` | path | - | Request payload file | | `--output` | path | - | Output file for results | | `--format` | string | `text` | Output format: text, json, html |

Examples

API Endpoint Benchmark

aqe benchmark api --target /api/users --iterations 1000

Benchmarks API endpoint with 1000 requests.

Database Query Benchmark

aqe benchmark database --target user-queries --concurrency 10

Benchmarks database queries with 10 concurrent connections.

Function Benchmark

aqe benchmark function --target calculateTotal --iterations 10000

Benchmarks function performance with 10,000 iterations.

Compare with Baseline

aqe benchmark api --target /api/orders --baseline baseline.json

Compares current performance against saved baseline.

Load Testing

aqe benchmark api --target /api/search --concurrency 50 --duration 60

Load tests endpoint with 50 concurrent users for 60 seconds.

Integration with Claude Code

Spawning Benchmark Agent

// Use Claude Code's Task tool to spawn the benchmark agent
Task("Execute performance benchmark", `
  Run comprehensive performance benchmarks:
  - Target: All API endpoints
  - Iterations: 5000 per endpoint
  - Concurrency: 20
  - Compare with baseline from last release

  Identify performance regressions and bottlenecks.
  Store results in memory: aqe/benchmarks/{bench-id}
`, "qe-performance-tester")

Comprehensive Benchmark Workflow

// Benchmark multiple targets in parallel
[Single Message]:
  Task("Benchmark API", "Test all API endpoints", "qe-performance-tester")
  Task("Benchmark Database", "Test critical queries", "qe-performance-tester")
  Task("Benchmark Functions", "Test core algorithms", "qe-performance-tester")

  TodoWrite({ todos: [
    {content: "Benchmark API endpoints", status: "in_progress", activeForm: "Benchmarking API"},
    {content: "Benchmark database queries", status: "in_progress", activeForm: "Benchmarking DB"},
    {content: "Benchmark core functions", status: "in_progress", activeForm: "Benchmarking functions"},
    {content: "Compare with baseline", status: "pending", activeForm: "Comparing baseline"},
    {content: "Generate performance report", status: "pending", activeForm: "Generating report"}
  ]})

Agent Coordination

Primary Agent

• **qe-performance-tester**: Main agent responsible for benchmarking

Supporting Agents

• **qe-test-executor**: Orchestrates benchmark execution
• **qe-quality-gate**: Validates against SLAs

Coordination Flow

1. Pre-Benchmark Hook
   ├─> Load baseline metrics (if provided)
   ├─> Configure benchmark parameters
   ├─> Warm up system
   └─> Establish monitoring

2. Benchmark Execution
   ├─> Run warmup iterations
   ├─> Execute benchmark iterations
   ├─> Collect timing metrics
   ├─> Monitor resource usage
   └─> Detect anomalies

3. Results Analysis
   ├─> Calculate statistics (mean, median, P95, P99)
   ├─> Compare with baseline (if provided)
   ├─> Identify regressions
   ├─> Detect bottlenecks
   └─> Generate recommendations

4. Post-Benchmark Hook
   ├─> Store benchmark results
   ├─> Update performance trends
   ├─> Train neural patterns
   └─> Notify fleet of findings

Memory Operations

Input Memory Keys

# Retrieve baseline metrics
npx claude-flow@alpha memory retrieve --key "aqe/benchmarks/baseline"

# Retrieve SLA thresholds
npx claude-flow@alpha memory retrieve --key "aqe/performance/sla"

# Retrieve historical benchmarks
npx claude-flow@alpha memory retrieve --key "aqe/benchmarks/history"

Output Memory Keys

# Store benchmark results
npx claude-flow@alpha memory store \
  --key "aqe/benchmarks/${bench_id}" \
  --value '{"mean": 45, "median": 42, "p95": 120, "p99": 250}'

# Update performance trends
npx claude-flow@alpha memory store \
  --key "aqe/performance/trends" \
  --value '[{"date": "2025-09-30", "mean": 45}]'

# Store baseline for future comparisons
npx claude-flow@alpha memory store \
  --key "aqe/benchmarks/baseline" \
  --value '{"mean": 45, "p95": 120, "timestamp": "2025-09-30T10:30:00Z"}'

Hooks and Coordination

Pre-Task Hook

npx claude-flow@alpha hooks pre-task \
  --description "Benchmark: ${target}" \
  --agent "qe-performance-tester"

Post-Task Hook

npx claude-flow@alpha hooks post-task \
  --task-id "${BENCH_ID}" \
  --results "${BENCH_RESULTS}"

Notify Fleet

npx claude-flow@alpha hooks notify \
  --message "Benchmark ${target} completed: $(echo "${BENCH_RESULTS}" | jq -r '.mean')ms mean"

Expected Outputs

API Benchmark Output

⚡ Running benchmark: /api/users
   Iterations: 1000
   Concurrency: 1
   Warmup: 100

🔥 Warming up (100 requests)...
   Warmup complete (avg: 48ms)

🚀 Running benchmark...
Progress: [████████████████████] 1000/1000

📊 Benchmark Results:

   Response Times:
   • Mean:     45ms
   • Median:   42ms
   • P95:      120ms
   • P99:      250ms
   • Min:      28ms
   • Max:      580ms

   Throughput:
   • Requests/sec: 22.2
   • Total time:   45.0s

   Status