/caching-strategies

name: caching-strategies
description: Deep reference for caching — what to cache, cache-aside vs read/write-through/write-behind, TTLs with jitter, eviction (LRU/LFU/FIFO), invalidation, and surviving stampedes (thundering herd / dogpile). Worked examples and a runnable jitter check.
type: skill
version: 2.0.0
updated: 2026-06-29

Caching Strategies

Caching is the cheapest order-of-magnitude performance win available, and the fastest way to serve *confidently wrong* data. The hard part was never reading from a cache — it's **invalidation**, **consistency under concurrency**, and **what happens the moment the cache is cold or wrong**. This skill is the deep reference: the patterns, the decisions, the trade-offs, and the failure modes that page you at 3am. Heavy detail lives in `references/`; copy-paste material in `examples/`; a runnable check in `scripts/`.

Mental model

A cache is a **bet**: you trade memory and a correctness risk for latency and load reduction. Every caching decision is answering four questions, in order:

| Question | What it decides | |---|---| | What is hot and tolerant of staleness? | *whether* to cache at all | | How does data get *into* the cache? | the **read/write pattern** (cache-aside, write-through…) | | How does stale data get *out*? | **TTL + invalidation** | | What happens on a miss storm or a dead cache? | **stampede control + fallback** |

If you can't answer the third and fourth questions before you ship, you don't have a caching strategy — you have a future incident. Decide the invalidation plan *first*.

1. What to cache

Cache data that is **read-often, expensive to produce, and tolerant of some staleness**. Good candidates: rendered product pages, the result of an expensive aggregation, a third-party API response, a permission lookup hit on every request. Bad candidates: a user's current account balance, a one-time-read report, anything where serving a 30-second-old value is a correctness or compliance bug.

Quantify it before caching. The benefit is roughly `hitRate × costPerMiss`. A 50% hit rate on a 200ms query is enormous; a 50% hit rate on a 2ms query is noise that you've paid for with a consistency risk. Measure hit rate in production — a cache below ~80% hit rate for point lookups usually means the key space is too sparse or the TTL is too short.

2. Read & write patterns

The pattern is *how data flows between your app, the cache, and the source of truth*. Pick one deliberately per dataset.

• **Cache-aside (lazy loading)** — the default. App checks cache; on miss, loads from the source and

populates the cache. Simple, resilient (a dead cache just means slow, not broken), but the first read of every key is a miss.

• **Read-through** — the cache library itself loads from the source on a miss. Same shape as

cache-aside but the loading logic lives behind the cache abstraction.

• **Write-through** — writes go to cache *and* source synchronously. Cache is always fresh; writes

are slower.

• **Write-behind (write-back)** — writes hit the cache and are flushed to the source asynchronously.

Fast writes, but you risk data loss on a crash before flush.

Full comparison with sequence diagrams and when each one bites: [references/cache-patterns.md](references/cache-patterns.md).

// Cache-aside — the 90% pattern. Note the explicit TTL and miss-population.
async function getProduct(id, { cache, db, ttl = 300 }) {
  const key = `product:${id}`;
  const hit = await cache.get(key);
  if (hit !== null && hit !== undefined) return JSON.parse(hit);   // cache hit

  const product = await db.getProduct(id);                          // miss -> source of truth
  if (product) await cache.set(key, JSON.stringify(product), { ex: ttl });
  return product;
}

3. TTL, eviction & jitter

A cache is finite, so entries leave in two ways: **TTL expiry** (time-based) and **eviction** (space pressure). Tune both.

• **TTL** bounds staleness. Short TTL = fresher + more misses; long TTL = fewer misses + staler data.
• **Eviction policy** decides who gets dropped when memory is full: **LRU** (evict least recently

used — the sane default), **LFU** (least frequently used — better for skewed popularity), **FIFO** (rarely what you want). See `references/eviction-and-ttl.md`.

• **TTL jitter is mandatory at scale.** If you populate 10,000 keys in a batch with a fixed 300s TTL,

they all expire in the same second and stampede the database together. Add randomness so expiries spread out.

// Add ±10% jitter so a batch of keys never expires in lockstep.
function jitteredTtl(baseSeconds, jitterRatio = 0.1) {
  const delta = baseSeconds * jitterRatio;
  const offset = (Math.random() * 2 - 1) * delta;          // uniform in [-delta, +delta]
  return Math.max(1, Math.round(baseSeconds + offset));
}
// jitteredTtl(300) -> ~270..330, never the same instant across a batch

`scripts/ttl-jitter.mjs` is a runnable, self-testing version of this function.

4. Invalidation

> "There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

TTL alone is *eventual* freshness. When correctness matters, invalidate **on write**:

• **Explicit bust** — on update, delete the key (`cache.del(key)`) so the next read repopulates. With

cache-aside this is the standard move. Beware the race: delete *after* the DB commits, not before.

• **Write-through** — keep the cache correct on the write path so there's nothing to bust.
• **Versioned keys** — embed a version/etag in the key (`product:42:v7`); bumping the version

instantly orphans all old entries without a scan.

The subtle bugs (delete-before-commit races, distributed invalidation lag, cache-vs-DB ordering) are covered in `references/invalidation-and-stampede.md`.

5. Stampede control (thundering herd / dogpile)

When a hot key expires, every concurrent request misses at once and they *all* hit the source — the **thundering herd**. A single popula