/elasticsearch-index-design

name: elasticsearch-index-design
description: >
  Design and review Elasticsearch index mappings for stated access patterns: correct
  field types, text+keyword multi-fields, doc_values tuning, mapping-explosion avoidance,
  and explicit shard settings. Use when creating a new index, reviewing a mapping
  for storage or query performance, fixing wrong field types, or when the user asks
  which type to use for search, filter, sort, or aggregation on a field.
metadata:
  author: elastic
  version: 0.1.0
  universal: true
compatibility: Elasticsearch 8.x or 9.x, self-managed, Elastic Cloud Hosted, or Elastic
  Cloud Serverless; explicit shard and replica settings apply to self-managed and
  Elastic Cloud Hosted only (managed internally on Serverless). Requires the `elastic`
  CLI ≥ 0.2 with `stack es` support.

Elasticsearch Index Design

Design explicit index mappings from access patterns, review existing mappings for type and storage mistakes, and apply corrections through a new index plus reindex when field types must change.

<!-- begin-partial: preamble -->

Environment Configuration

This skill executes Elasticsearch operations through the `elastic` CLI. If the [`elastic` CLI](https://github.com/elastic/cli#configuration) is not installed, tell the user what it is needed for. Do not guess credentials, call the HTTP API directly, or attempt other workarounds.

This skill references operations in HTTP-shorthand form (e.g., `GET /`, `GET /_cat/indices`, `GET /{index}/_mapping`, `GET /{index}/_settings/index.mode`, `POST /_query`). The [Operations](#operations) table at the end of this document maps each shorthand to the equivalent `elastic` CLI command — always use the CLI rather than calling the HTTP API directly.

<!-- end-partial: preamble -->

Process

1. **Gather access patterns per field.** Before choosing types, list how each field is used. For every field capture:

• **Search** — full-text match, phrase, relevance scoring?
• **Filter** — exact term, terms set, prefix?
• **Aggregate** — terms, cardinality, histogram, stats?
• **Sort** — ascending/d descending in result sets?
• **Retrieve only** — returned in `_source` but never queried?

The decision: classify each field into one primary access pattern (search, exact, numeric metric, date, boolean, structured object, or retrieve-only). Missing access-pattern data is a blocker — ask the user rather than guessing. Call `GET /` to confirm connectivity; when reviewing an existing index, call `GET /{index}/_mapping` to ground the discussion in the current mapping.

2. **Choose field types from access patterns.** Map each field to the minimal type set that satisfies its pattern. Read [Field Type Decisions](references/field-type-decisions.md) and [Multi-Field Patterns](references/multi-field-patterns.md) before proposing mappings.

Key judgments:

| Pattern | Mapping | | ----------------------------------------------- | ------------------------------------------------------------ | | Full-text search only | `text` (no keyword sub-field) | | Filter / agg / sort only | `keyword` (not `text`) | | Full-text search **and** sort or aggregation | `text` with `fields.keyword` multi-field | | Decimal price or metric | `double`, `float`, or `scaled_float` — not `text` or integer | | Timestamp | `date` | | True/false flag | `boolean` | | Free-form key/value map with many distinct keys | `flattened` — not dynamic `object` |

**Multi-field rule:** When a field must be searchable **and** sortable/aggregatable (e.g. product `name`), map it as `text` with a `keyword` sub-field — search on `name`, sort and aggregate on `name.keyword`. Mapping as only `text` or only `keyword` is wrong for that combined pattern.

**Explicit mapping rule:** For new indices, always define mappings explicitly with `PUT /{index}`. Do not rely on dynamic mapping for production indices — the first document can lock in wrong types (strings as `text`, ambiguous numbers as `keyword`).

**Index settings:** Set deliberate `number_of_shards` and `number_of_replicas` in the same `PUT /{index}` request when the deployment allows it (Self-Managed / Elastic Cloud Hosted). On Serverless, omit shard and replica counts (Elastic manages them); still supply explicit mappings. State chosen values or document that defaults apply.

Example — `products` index optimized for search plus sort/agg on name:

   {
     "settings": {
       "number_of_shards": 1,
       "number_of_replicas": 1
     },
     "mappings": {
       "properties": {
         "name": {
           "type": "text",
           "fields": {
             "keyword": { "type": "keyword", "ignore_above": 256 }
           }
         },
         "price": { "type": "double" },
         "created": { "type": "date" },
         "in_stock": { "type": "boolean" }
       }
     }
   }

Create with `PUT /products` passing the `settings` and `mappings` blocks. Verify with `GET /products/_mapping`.

3. **Guard against mapping explosion and storage bloat.** On high-volume indices, type mistakes multiply cost. Read [Mapping Explosion and Storage Bloat](references/mapping-explosion.md) and apply these review checks:

• **Analyzed-but-not-searched fields** — Fields used only for filter and aggregation (`url`, HTTP `status_code`,

`tags`, IDs) must be `keyword`, not `text`. `text` wastes space; aggregations on `text` require fielddata or a `.keyword` sub-field that