/distributed-search

name: distributed-search
description: This skill should be used when the user designs a "search system", needs "full-text search", asks about an "inverted index", "Elasticsearch / OpenSearch", "relevance ranking" (TF-IDF/BM25), "search autocomplete / typeahead", an "indexing pipeline", or "faceted search". It gives the crawl/index/search architecture, index sharding and replication, ranking, and near-real-time indexing. Use it whenever users must query text by relevance rather than fetch rows by key, even if they don't say "search engine".

Distributed search

Find the documents that best match a free-text query, ranked by relevance, fast, across more data than one machine holds. Getting it wrong means either slow `LIKE '%term%'` scans that melt the primary database, or a search box that returns the wrong results and erodes user trust — both are silent until traffic or corpus size exposes them.

When to reach for this

Users type words and expect ranked, relevant matches — not exact-key lookups. The corpus is text-heavy (documents, products, logs, messages), queries are ad-hoc (any term, any combination), and results need ranking, highlighting, facets, or typeahead. Reach for it when a `WHERE col LIKE` or full-table scan is already the read bottleneck, or when you need fuzzy/partial matching a B-tree index cannot serve.

When NOT to

The access pattern is fetch-by-known-key or a fixed filter — a primary database index serves that far more cheaply and consistently; keep it in `data-storage`. The corpus is tiny (thousands of rows): an in-process filter or the database's built-in full-text index is enough — a separate search cluster is pure operational overhead (YAGNI). Search is a *derived, eventually-consistent* copy of your data; never make it the system of record.

Clarify first

• **Corpus size and growth** — document count, average doc size, total index

bytes? (→ `back-of-the-envelope`) This decides shard count.

• **Query QPS and shape** — read-heavy? term queries, phrase, fuzzy, facets,

autocomplete? Latency target (p99)?

• **Indexing freshness** — must a new/edited doc be searchable in seconds

(near-real-time) or is minutes/hours of lag fine?

• **Relevance bar** — is exact term-match enough, or do users expect "best"

results (ranking, synonyms, typo tolerance)?

• **Write rate** — how many docs/sec change? This sizes the indexing pipeline.

The options

**The pipeline** (almost always present): a source emits document changes → an **indexing pipeline** transforms/analyzes them → the **inverted index** stores term→document postings → the **query path** matches and ranks. For a crawl-based system (web search), prepend crawl → parse → dedupe; that crawler is its own subsystem feeding the same pipeline.

**Index build mode**

• **Batch / bulk reindex:** rebuild the whole index periodically. Use when the

corpus changes slowly or freshness in hours is acceptable.

• **Near-real-time (incremental):** apply changes continuously so docs are

searchable in seconds. Use when users expect to find what they just wrote.

**Ranking model**

• **Boolean / filter only:** match, no scoring. Use for exact filtering (tags,

facets) where order doesn't matter.

• **TF-IDF / BM25 (lexical):** score by term frequency and rarity. The default

full-text relevance model; cheap and explainable.

• **Hybrid (lexical + signals):** blend BM25 with popularity, recency, or

business boosts. Use when "best" means more than word overlap.

**Autocomplete**

• **Prefix trie / FST in memory:** sub-millisecond typeahead from a prefix. Use

for suggestion-as-you-type.

• **Edge-n-gram index:** prefix matching inside the main index. Use when

suggestions must also respect filters/relevance, at higher cost.

**Distribution**: split the index into **shards** (each a self-contained inverted index over a doc subset) for capacity, and **replicas** per shard for read throughput and fault tolerance. Sharding theory lives in `data-storage`.

Trade-offs

| Option | What it solves | What it worsens | Change it when | |---|---|---|---| | Batch reindex | Simple, atomic swap, no live-write complexity | Stale until next build; full rebuild is costly | Users need fresh results → near-real-time | | Near-real-time | Seconds-fresh; no full rebuild | Segment churn, merge load, refresh cost on writes | Write rate or merge cost overwhelms nodes → batch/larger refresh interval | | Boolean/filter | Cheapest; deterministic | No notion of "best" result | Users judge result quality → add BM25 | | BM25 | Good relevance, explainable, cheap | Ignores popularity/recency/intent | Word-overlap isn't enough → hybrid signals | | Hybrid signals | Matches business/user intent | Complex, harder to debug, needs tuning data | Tuning cost exceeds value → fall back to BM25 | | Prefix trie/FST | Fastest typeahead | Separate structure to build/refresh; ignores filters | Suggestions need filters/relevance → edge-n-gram | | More shards | Parallelism, fits big corpus | Per-query fan-out + merge overhead; tiny shards waste resources | Fan-out latency dominates → fewer, larger shards | | More replicas | Read QPS + HA | More RAM/disk; replication lag on writes | Write amplification hurts → fewer replicas |

Behavior under stress

Search amplifies trouble through **fan-out** and **derived-data lag**.

• **Query fan-out tail latency:** every query hits all shards; the slowest shard

sets the response time. One hot or GC-paused shard drags every query. *Mitigate:* size shards evenly, add replicas, cap result depth, use timeouts + partial results.

• **Indexing vs query contention:** a write/merge surge (bulk import, reindex)

steals CPU and I/O from queries, spiking latency. *Mitigate:* throttle bulk indexing, schedule big merges off-peak, isolate index vs query node roles.

• **Hot shard / skew:** an uneven shard key concentrates docs or popular terms on

one node. *Mitigate:* hash-route documents; reroute or split the hot shard.

• *