/caching

name: caching
description: Use when adding or debugging a cache. Covers cache placement, invalidation strategies, stampede protection, TTL selection, and the consistency you are trading away. Trigger on "add redis caching", "users are seeing stale data", "cache stampede after expiry", "should we cache this". Do not use for browser caching headers and page load — see the web-performance skill; for a query that is slow at the source, see the database-performance skill.
metadata:
  category: backend
  version: 1.0.0
  tags: [caching, redis, invalidation, performance, stampede]

Caching

Purpose

Add a cache with a clear invalidation story, or find out why the existing one is serving stale data. A cache is a second source of truth; introducing one is a consistency decision, not just a performance one.

When to Use

• A read path is slow and the data is read far more often than it is written.
• An upstream dependency is expensive, rate-limited, or unreliable.
• Debugging stale reads, cache stampedes, or unexplained memory growth.

Capabilities

• Cache placement: client, CDN, application, database.
• Strategies: cache-aside, read-through, write-through, write-behind.
• Invalidation: TTL, explicit, event-driven, versioned keys.
• Stampede protection: locking, early recomputation, request coalescing.
• Negative caching and hot-key mitigation.

Inputs

• The read/write ratio and the cost of the uncached path.
• How stale the data is allowed to be — in seconds, agreed with the product owner.
• Cardinality of the key space and expected memory footprint.

Outputs

• A key schema, a TTL, and an invalidation trigger for each cached entity.
• Stampede protection on any key expensive enough to matter.
• A hit-rate metric, because a cache you do not measure is a cache you do not understand.

Workflow

1. **Prove the need** — Measure the uncached path first. A cache in front of a missing index is a permanent workaround for a five-minute fix. 2. **Decide the staleness budget** — Ask what breaks if a user sees data five seconds old. The answer determines the TTL and whether you need explicit invalidation. 3. **Design the key** — Include everything that varies the result: tenant, locale, permission scope. A key that omits a dimension leaks data between users. 4. **Choose invalidation** — TTL alone for tolerant data. TTL plus explicit deletion on write for data that must be fresh. Versioned keys when deletion is unreliable. 5. **Protect against stampedes** — When a hot key expires, every concurrent request misses simultaneously and hits the origin at once. Use a lock or single-flight. 6. **Measure** — Hit rate, latency at each layer, and eviction rate. Falling hit rate with rising memory means the key space is too large.

Best Practices

• Cache the expensive computation, not the whole response. Response-level caching multiplies the key space by every variant.
• A cache key missing the tenant or the user's permission scope is a data-leak vulnerability, and it will be found by a customer.
• Never cache with an unlimited TTL and rely on invalidation alone. Invalidation fails; the TTL is the safety net.
• Set memory limits and an eviction policy explicitly (`allkeys-lru`). A Redis instance with no `maxmemory` will be killed by the OOM killer.
• Do not cache errors indefinitely, but do cache "not found" briefly — otherwise a bad ID becomes an origin DoS.
• Warm caches before cutting traffic over. A cold cache under full load is an outage.

Examples

**Single-flight cache-aside, preventing a stampede:**

async def get_pricing(tenant_id: str, sku: str) -> Pricing:
    key = f"pricing:v3:{tenant_id}:{sku}"

    if (cached := await redis.get(key)) is not None:
        return Pricing.model_validate_json(cached)

    # Only one caller per key computes; the rest wait for the result.
    lock_key = f"{key}:lock"
    if await redis.set(lock_key, "1", nx=True, ex=10):
        try:
            pricing = await compute_pricing(tenant_id, sku)      # expensive
            await redis.set(key, pricing.model_dump_json(), ex=300)
            return pricing
        finally:
            await redis.delete(lock_key)

    # Lost the race: wait briefly for the winner, then fall back to computing.
    for _ in range(20):
        await asyncio.sleep(0.05)
        if (cached := await redis.get(key)) is not None:
            return Pricing.model_validate_json(cached)

    return await compute_pricing(tenant_id, sku)

Notes

• The `v3` in the key is a schema version. When the cached shape changes, bump it — this invalidates the whole namespace atomically without a scan-and-delete.
• Adding jitter to TTLs (`ex=300 + random.randint(0, 60)`) prevents a cohort of keys written together from expiring together.
• A 95% hit rate sounds excellent, but if the 5% of misses are all on the single hottest key, the origin still sees the full load. Look at misses per key, not just the aggregate.