/elasticsearch-reindex

name: elasticsearch-reindex
description: >
  Guide Elasticsearch reindex for performance: local and remote, slicing, throttling,
  task API. Use when copying or migrating indices, changing mappings, or transforming
  during reindex.
metadata:
  author: elastic
  version: 0.2.0
  universal: true
compatibility: Elasticsearch 8.x or 9.x, self-managed, Elastic Cloud Hosted, or Elastic
  Cloud Serverless; local reindex works on all deployment types, while remote reindex
  requires host allowlisting on self-managed / Elastic Cloud Hosted (Serverless supports
  Elastic Cloud Hosted remotes only, Tech Preview). Requires the `elastic` CLI ≥ 0.2
  with `stack es` support.

Elasticsearch Reindex

Copy documents from source indices or data streams to a destination using `POST /_reindex`. An expert reindex workflow prepares the destination explicitly, chooses local versus remote execution, filters at the source when only a subset is needed, runs long copies asynchronously, tracks the task to completion, and verifies the destination document count before reporting results.

<!-- begin-partial: preamble -->

Environment Configuration

This skill executes Elasticsearch operations through the `elastic` CLI. If the [`elastic` CLI](https://github.com/elastic/cli#configuration) is not installed, tell the user what it is needed for. Do not guess credentials, call the HTTP API directly, or attempt other workarounds.

This skill references operations in HTTP-shorthand form (e.g., `GET /`, `GET /_cat/indices`, `GET /{index}/_mapping`, `GET /{index}/_settings/index.mode`, `POST /_query`). The [Operations](#operations) table at the end of this document maps each shorthand to the equivalent `elastic` CLI command — always use the CLI rather than calling the HTTP API directly.

<!-- end-partial: preamble -->

Process

1. **Confirm connectivity and deployment type.** Call `GET /`. Read `build_flavor` and `version.number` to know whether shard, replica, and cluster-settings APIs are available (Serverless manages shards/replicas internally and blocks most `_cluster/*` APIs). The decision: continue only when the cluster is reachable. If the call fails, stop — do not guess endpoints or credentials.

2. **Decide local versus remote reindex.** Compare where the source and destination live.

• **Same cluster** — use local reindex: `source.index` and `dest.index` only. Do **not** add `source.remote` when

both indices are on the cluster you are connected to.

• **Different cluster** — use reindex from remote: add `source.remote` with the remote cluster URL and credentials.

Remote reindex does not support slicing; compensate with query-based partitioning (date ranges, term filters) across parallel requests. Confirm the remote host is allowlisted on Self-Managed / ECH (`reindex.remote.whitelist` in cluster config); Serverless manages allowlisting internally (ECH remotes only, Tech Preview).

Data needed: source index name(s), destination index name, and whether they share a cluster.

3. **Inspect the source — never guess field names or counts.** Call `GET /{source}/_mapping` to ground field names and types. Call `GET /{source}/_count` (or `GET /_cat/count/{source}?h=count` on Self-Managed / ECH) to learn how many documents exist.

The decision: **full copy** versus **filtered subset**.

• Full copy — omit `source.query` (match-all behavior).
• Filtered subset — add `source.query` with Query DSL. For time ranges, use a `range` filter on the timestamp field

(commonly `@timestamp`), e.g. `"gte": "2025-01-01", "lt": "2025-02-01"` for January 2025. Do **not** run a full-index copy when the user asked for a date range or other filter.

Data needed: the user's filter criteria and the mapping-confirmed field names.

4. **Prepare the destination index before copying.** `_reindex` does **not** copy mappings, shard counts, or analyzers. Create the destination with explicit settings and mappings derived from the source mapping via `PUT /{dest}`.

• On Self-Managed / ECH: set `number_of_replicas: 0` and `refresh_interval: "-1"` on the destination during the copy

for write throughput; restore production values afterward with `PUT /{dest}/_settings`.

• On Serverless: omit `number_of_shards` and `number_of_replicas` (managed by Elastic); you may set

`refresh_interval: "-1"` during the copy.

• For **data stream** destinations: ensure an index template with `data_stream: {}` exists, create the data stream,

and set `dest.op_type` to `"create"` (append-only).

The decision: create/prepare the target rather than relying on auto-creation with dynamic mapping. Wrong or missing mappings cause partial failures or silent type coercion.

Data needed: destination name, corrected or compatible mappings, and deployment-specific settings constraints.

5. **Build and submit the reindex request.** Call `POST /_reindex?wait_for_completion=false` for any copy that may take more than a few seconds or when the user says the index is large — the response returns a **task id** immediately instead of blocking.

Request body essentials:

• `source.index` — source index or data stream (correct name, not reversed with `dest.index`).
• `dest.index` — prepared destination from step 4.
• `source.query` — include only when step 3 chose a filtered subset.
• `conflicts: "proceed"` — when retrying a partially complete reindex.
• Optional tuning: `source.size` (batch size), `requests_per_second` (throttle), `slices=auto` on local reindex only

(parallelize per primary shard — never for remote), `scroll` (increase keep-alive on slow clusters), `max_docs` (test runs), `script` (transform), `dest.pipeline` (ingest enrichment).

Example filtered subset (January 2025 only):

   {
     "source": {
       "index": "eval-reindex-src",
       "query": {
         "range": {
           "@timestamp": { "gte": "2025-01-01", "lt": "20