/system-design-building-blocks

name: system-design-building-blocks
description: "Select infrastructure components by the constraint each removes: load balancers, caches and invalidation, queues and pub/sub, CDN, API gateway, rate limiting, consistent hashing. Use when choosing cache, queue, replica, or edge layers; defer engine tuning to the database category."
metadata:
  triggers:
    keywords:
      - load balancer
      - cache strategy
      - message queue
      - cdn
      - api gateway
      - consistent hashing
      - pub/sub
      - rate limiter
      - backpressure

System Building Blocks

**Priority: P1 (HIGH)**

Add a component only to remove a named constraint. Every component adds failure modes and cost.

Selection Table

| Constraint observed | Component | Cost accepted | | --- | --- | --- | | One instance saturates CPU or connections | Horizontal scaling behind a load balancer | Statelessness required; session moves to a store | | Read QPS exceeds store ceiling; data tolerates staleness | Read-through cache | Stale reads up to TTL; invalidation complexity | | Write spikes exceed sustainable store throughput | Queue plus async worker | Eventual completion; needs idempotent consumers | | Slow non-critical work blocks the request path | Background job or event | Extra delivery guarantees to reason about | | Static or media bytes dominate egress | CDN | Cache purge lag; origin still needed for misses | | Cross-cutting authn, quota, routing repeated per service | API gateway | Central chokepoint; must be HA | | One tenant or key can exhaust capacity | Rate limiter or admission control | Rejections need clear client contracts | | Cache or shard nodes change count during scaling | Consistent hashing | Rebalancing logic; hot-key skew persists |

Caching Rules

• Default to **cache-aside** for read-heavy paths; use write-through only when read-after-write must be exact.
• Set TTL from tolerable staleness, not from convenience. Every cached key needs a stated TTL and an invalidation trigger.
• Guard the miss path: single-flight or request coalescing prevents a stampede when a hot key expires.
• Never cache what you cannot invalidate. Never cache per-user data under a shared key.

Queue Rules

• Choose delivery semantics explicitly: at-least-once plus idempotent consumers is the safe default.
• Partition by the entity that must stay ordered; ordering is per partition, never global.
• Size the queue for the burst, and define the drain rate. An unbounded queue converts a throughput problem into a latency and memory problem.
• Apply backpressure at the producer when depth exceeds the drain budget; shedding beats silent growth.

Session State

• Default to a stateless token (signed JWT or opaque id verified at the edge); the app tier keeps nothing per user.
• Move to a shared session store (Redis) when sessions must be revocable immediately or carry more than a token can hold.
• Sticky sessions are a last resort: they survive one deploy badly and block scale-in. Never store session state in app memory.

Edge and Traffic Rules

• L4 balances connections, L7 routes on content; use L7 when routing needs paths, headers, or canaries.
• Health checks must test a dependency-aware readiness endpoint, not a static 200.
• Rate limit at the edge by identity and by cost class; return a retry hint the client can honor.

Anti-Patterns

• **No component without a constraint**: a box that solves no stated problem gets removed.
• **No cache as a correctness fix**: caching hides a slow query; fix the query, then cache.
• **No queue without a consumer budget**: define drain rate and DLQ before shipping.
• **No shared mutable session in app memory**: it blocks horizontal scaling.

References

• [Building Blocks Detail](references/building-blocks.md) - per-component decision notes, invalidation patterns, hashing mechanics