/condition-based-waiting

name: condition-based-waiting
description: "Polling, retry, and backoff patterns."
user-invocable: false
allowed-tools:
  - Read
  - Write
  - Bash
  - Grep
  - Glob
  - Edit
routing:
  triggers:
    - "exponential backoff"
    - "health check polling"
    - "retry pattern"
    - "wait for"
    - "keep trying until"
    - "poll until ready"
    - "retry until success"
  category: process
  pairs_with:
    - shell-process-patterns
    - service-health-check

Condition-Based Waiting

Implement condition-based polling and retry patterns with bounded timeouts, jitter, and error classification. Select the right pattern for the scenario, implement it with safety bounds, and verify both success and failure paths.

| Pattern | Use When | Key Safety Bound | |---------|----------|-----------------| | Simple Poll | Wait for condition to become true | Timeout + min poll interval | | Exponential Backoff | Retry with increasing delays | Max retries + jitter + delay cap | | Rate Limit Recovery | API returns 429 | Retry-After header + default fallback | | Health Check | Wait for service(s) to be ready | All-pass requirement + per-check status | | Circuit Breaker | Prevent cascade failures | Failure threshold + recovery timeout |

Reference Loading Table

| Signal | Load These Files | Why | |---|---|---| | reviewing wait/retry code for polling, retry, and backoff mistakes | `preferred-patterns.md` | Loads detailed guidance from `preferred-patterns.md`. | | writing wait code: polling, exponential backoff, rate-limit recovery, health checks, circuit breaker | `implementation-patterns.md` | Loads detailed guidance from `implementation-patterns.md`. | | tests, implementation patterns | `testing-patterns.md` | Loads detailed guidance from `testing-patterns.md`. |

Instructions

Before implementing any pattern, read the repository CLAUDE.md and search the codebase for existing wait/retry patterns to maintain consistency with what already exists.

Step 1: Select the Pattern

Walk this decision tree to pick the right pattern. Only implement the pattern directly needed -- do not add circuit breakers when simple retries suffice, and do not add health checks when a single poll works.

1. Waiting for a condition to become true?
   YES -> Simple Polling (Step 2)
   NO  -> Continue

2. Retrying a failing operation?
   YES -> Rate-limited (429)?
         YES -> Rate Limit Recovery (Step 5)
         NO  -> Exponential Backoff (Step 4)
   NO  -> Continue

3. Waiting for a service to start?
   YES -> Health Check Waiting (Step 6)
   NO  -> Continue

4. Service frequently failing, need fast-fail?
   YES -> Circuit Breaker (Step 7)
   NO  -> Simple Poll or Backoff

Step 2: Implement Simple Polling

Wait for a condition to become true with bounded timeout.

1. Define the condition function (returns truthy when ready). 2. Set timeout and poll interval based on target type. Use `time.monotonic()` for elapsed time measurement -- never `time.time()`, which drifts with clock adjustments.

| Target Type | Min Interval | Typical Interval | Example | |-------------|-------------|-----------------|---------| | In-process state | 10ms | 50-100ms | Flag, queue, state machine | | Local file/socket | 100ms | 500ms | File exists, port open | | Local service | 500ms | 1-2s | Database, cache | | Remote API | 1s | 5-10s | HTTP endpoint, cloud service |

Never busy-wait (tight loop with no sleep). The minimum poll interval is 10ms for local operations, 100ms for external services. Tighter loops burn CPU, cause thermal throttling, and starve other processes.

3. Implement with a mandatory timeout. Every wait loop must have a maximum timeout to prevent infinite hangs. The timeout error message must include what was waited for and the last observed state so the caller can diagnose failures.

# Core pattern (full implementation in references/implementation-patterns.md)
start = time.monotonic()
deadline = start + timeout_seconds
while time.monotonic() < deadline:
    result = condition()
    if result:
        return result
    time.sleep(poll_interval)
raise TimeoutError(f"Timeout waiting for: {description}")

4. Report wait progress with timeout values and retry counts during the wait. Cancel pending operations when the timeout expires. 5. Test with both success and timeout scenarios. Force the condition to never become true and confirm TimeoutError fires with a descriptive message.

Step 3: Verify Before Proceeding

After implementing any pattern from Steps 2-7, verify:

• Success path works as expected
• Failure/timeout path produces a descriptive error
• Logging captures each attempt with failure reason and attempt number
• No arbitrary sleep values remain (replace `sleep(N)` with condition-based polling)

Step 4: Implement Exponential Backoff

Retry failing operations with increasing delays and jitter.

1. Classify errors before implementing retries. Separate transient from permanent errors -- retrying permanent errors wastes time and quota.

• **Retryable**: 408, 429, 500, 502, 503, 504, network timeouts, connection refused
• **Non-retryable**: 400, 401, 403, 404, validation errors, auth failures

2. Configure backoff parameters. Every retry loop must have a maximum retry count.

• `max_retries`: 3-5 for APIs, 5-10 for infrastructure
• `initial_delay`: 0.5-2s
• `max_delay`: 30-60s
• `jitter_range`: 0.5 (adds +/-50% randomness)

3. Implement with jitter. Jitter is mandatory on all exponential backoff -- without it, all clients retry at the same instant after an outage (thundering herd), amplifying the load spike that caused the failure.

# Core pattern (full implementation in references/implementation-patterns.md)
for attempt in range(max_retries + 1):
    try:
        return operation()
    except retryable_exceptions as e:
        if attempt >= max_retries:
            raise
        jitter = 1.0 + random.uniform(-0.5, 0.5)
        actual_delay = min(delay * jitt