/api-search-elasticsearch

name: api-search-elasticsearch
description: Elasticsearch patterns -- client setup, index management, search DSL, aggregations, vector search, bulk operations, deep pagination

Elasticsearch Patterns

> **Quick Guide:** Use `@elastic/elasticsearch` (v8.x/v9.x) as the TypeScript client. Elasticsearch is **near real-time** -- documents are NOT searchable immediately after indexing; they become visible after a refresh (default: every 1 second on active indices). You MUST define explicit mappings before indexing -- dynamic mapping infers types from the first document, and mismatched types in later documents cause hard failures you cannot fix without reindexing. Use `search_after` + Point in Time (PIT) for deep pagination -- NOT `from`/`size` beyond 10,000 hits and NOT the scroll API (deprecated for search). Use the `bulk` API or `client.helpers.bulk()` for any batch operation -- never loop individual index calls.

---

<critical_requirements>

CRITICAL: Before Using This Skill

**(You MUST define explicit index mappings BEFORE indexing documents -- dynamic mapping infers types from the first document, and if a later document sends a different type for the same field, indexing fails with a `mapper_parsing_exception` that CANNOT be fixed without reindexing into a new index)**

**(You MUST use the `bulk` API for batch operations -- looping individual `client.index()` calls is orders of magnitude slower and can overwhelm the cluster with HTTP connections)**

**(You MUST NOT use `from`/`size` pagination beyond 10,000 results -- Elasticsearch throws `Result window is too large` by default; use `search_after` + PIT instead)**

**(You MUST NOT use `refresh: true` or `refresh: "wait_for"` in production request handlers -- forcing a refresh on every write degrades cluster performance; let the default 1-second refresh interval handle it)**

</critical_requirements>

---

Examples

• [Core Patterns](examples/core.md) -- Client setup, index management, document CRUD, search basics, TypeScript integration
• [Aggregations](examples/aggregations.md) -- Terms, range, date_histogram, nested, pipeline aggregations
• [Vector Search](examples/vector-search.md) -- Dense vector fields, kNN queries, hybrid search, similarity metrics
• [Pagination](examples/pagination.md) -- from/size, search_after, Point in Time, scroll helpers
• [Bulk Operations](examples/bulk-operations.md) -- Bulk API, bulk helper, reindexing patterns

**Additional resources:**

• [reference.md](reference.md) -- Search DSL cheat sheet, mapping types, aggregation reference, decision frameworks

---

**Auto-detection:** Elasticsearch, elasticsearch, @elastic/elasticsearch, client.search, client.index, client.bulk, client.indices.create, client.indices.putMapping, dense_vector, knn, search_after, point in time, openPIT, aggregations, aggs, bool query, match query, term query, multi_match, nested query, range query, client.helpers.bulk, client.helpers.scrollSearch, BulkResponse, SearchResponse, MappingProperty

**When to use:**

• Full-text search with advanced relevance tuning (BM25, custom analyzers, boosting)
• Aggregations and analytics (terms, histograms, pipeline aggregations)
• Vector/semantic search with kNN on dense_vector fields
• Log and event data search with time-based queries
• Complex structured queries combining bool, nested, range, and geo filters
• Search across large datasets requiring deep pagination (search_after + PIT)

**Key patterns covered:**

• Client initialization and connection management
• Index management with explicit mappings and settings
• Document CRUD (index, get, update, delete)
• Search DSL (match, term, bool, range, nested, multi_match)
• Aggregations (terms, range, date_histogram, nested, pipeline)
• Full-text analysis (custom analyzers, tokenizers, filters)
• Vector search (dense_vector, kNN, hybrid text+vector)
• Bulk operations and reindexing
• Deep pagination (search_after + PIT)

**When NOT to use:**

• Simple keyword search on small datasets (client-side filtering or database LIKE queries are simpler)
• Primary data store (Elasticsearch is a search engine, not a database -- always have a source of truth elsewhere)
• Strong consistency requirements (Elasticsearch is eventually consistent by design)
• Simple autocomplete on a small list (a prefix trie or client-side filter is simpler)

---

<philosophy>

Philosophy

Elasticsearch is a **distributed search and analytics engine** built on Apache Lucene. It excels at full-text search, structured queries, aggregations, and vector search at scale. Core principles:

1. **Near real-time, not real-time** -- Documents are indexed into segments. A refresh (default: every 1 second on active indices) makes new segments searchable. Do not expect immediate consistency after writes. 2. **Mappings are immutable** -- Once a field type is set (text, keyword, integer, etc.), it cannot be changed. Wrong types require reindexing into a new index. Always define mappings explicitly before first document. 3. **Search engine, not database** -- Elasticsearch should not be your source of truth. Always have a primary database and sync to Elasticsearch for search. 4. **Bulk everything** -- The bulk API amortizes HTTP overhead across thousands of operations. Never loop individual index/update/delete calls. 5. **Pagination has limits** -- `from`/`size` is capped at 10,000 hits by default (`index.max_result_window`). Deep pagination requires `search_after` + Point in Time (PIT). The scroll API is deprecated for search use cases. 6. **Text vs keyword matters** -- `text` fields are analyzed (tokenized, lowercased) for full-text search. `keyword` fields are exact-match only. Getting this wrong means either broken search or broken aggregations/filters.

</philosophy>

---

<patterns>

Core Patterns

Pattern 1: Client Setup

Initialize the client with node URL and authentication. The client supports Elastic Cloud, API keys, basic auth, and bearer tokens.