/ml-integrations

name: security-ml-integrations
description: >
  Install and manage Elastic security ML integration packages (beaconing, ded, dga,
  lmd, pad, problemchild). Handles the full lifecycle: pre-flight checks, Fleet package
  install, component templates, ingest pipelines, transforms, data views, ML anomaly
  detection jobs, and detection rules. Use when a user asks to install, set up, check
  the status of, or troubleshoot a security ML integration package.
metadata:
  author: elastic
  version: 0.1.0
  universal: true
compatibility: Requires Kibana 9.4+ with Fleet, ML, and Security; a Platinum, Enterprise,
  or trial license (or Serverless); manage_ml; and the elastic CLI ≥ 0.5.

Security ML Integration Package Management

Install, check, or troubleshoot a security ML package. Supported packages: `beaconing` (Network Beaconing Identification), `ded` (Data Exfiltration Detection), `dga` (Domain Generation Algorithm Detection), `lmd` (Lateral Movement Detection), `pad` (Privileged Access Detection), `problemchild` (Living off the Land Attack Detection).

<!-- begin-partial: preamble -->

Environment Configuration

This skill executes Elasticsearch operations through the `elastic` CLI. If the [`elastic` CLI](https://github.com/elastic/cli#configuration) is not installed, tell the user what it is needed for. Do not guess credentials, call the HTTP API directly, or attempt other workarounds.

This skill references operations in HTTP-shorthand form (e.g., `GET /`, `GET /_cat/indices`, `GET /{index}/_mapping`, `GET /{index}/_settings/index.mode`, `POST /_query`). The [Operations](#operations) table at the end of this document maps each shorthand to the equivalent `elastic` CLI command — always use the CLI rather than calling the HTTP API directly.

<!-- end-partial: preamble -->

Package source patterns, shared template fields, and ingest-pipeline wiring live in [package capabilities](references/package-capabilities.md).

Process

1. **Classify the request.** Status, README, or install. Status and README stay read-only — do not install, start transforms, deploy jobs, or enable rules. After identifying the package, branch: status uses `GET kbn:/api/fleet/epm/packages/{package}` only (title, version, status); README adds `GET kbn:/api/fleet/epm/packages/{package}/{version}/docs/README.md` and returns that document; install continues.

2. **Identify the package.** Map the user's words to one of the six package names above. If the name is ambiguous, ask before calling any API. Load that package's row from [package capabilities](references/package-capabilities.md) — omit fields that are empty for that package.

3. **Gather environment state before any mutation.** Install only. `GET /_license` first — continue only for `platinum`, `enterprise`, `trial`, or Serverless. If `GET /_ml/info` returns 403, report missing `manage_ml` and stop. Then collect:

• Package: `GET kbn:/api/fleet/epm/packages/{package}` for name, title, version, and `status`.
• Source data: `POST /{source_patterns}/_search` with `size: 0` and `track_total_hits: true`. If every pattern has

zero hits, warn that jobs will not produce useful anomalies.

• Shared template (DGA, PAD, ProblemChild only): `GET /_component_template/{name}`. Exists → plan a merge of the

fields in the reference. 404 → plan a create with only those fields.

• Shared pipeline (DGA, PAD, ProblemChild only): `GET /_ingest/pipeline/{name}`. Exists → plan to append a processor.

404 → plan a single-processor pipeline.

• Transforms: `GET /_transform/{pattern}` (skip if the package has no transform pattern). Record each `id`,

`dest.index`, and the `.all` alias (`dest.aliases` where `move_on_creation` is false).

• Data views: `GET kbn:/api/data_views` and check whether the package's data-view pattern already exists.
• ML capacity: `GET /_ml/info` (total memory) and `GET /_ml/anomaly_detectors/_all/_stats` (sum memory of **opened**

jobs). `GET kbn:/internal/ml/modules/get_module/{module_id}` for job count and per-job memory. If jobs do not fit, say so and suggest prioritizing job groups — do not deploy anyway without the user accepting the risk.

• Rules: `GET kbn:/api/detection_engine/rules/prepackaged/_status` and

`POST kbn:/internal/detection_engine/prebuilt_rules/installation/_review` filtered by the package detection tag.

4. **Present a plan and wait for confirmation.** Show package identity and status, source-data hits (warn if all zero), template/pipeline merge-or-create, transform count and states, data-view presence, ML memory fit, and rule count. **Do not call any write API until the user confirms.** If they decline, stop.

5. **Install the Fleet package, then apply shared mappings.** After confirmation:

• `POST kbn:/api/fleet/epm/packages/{package}` to install assets (transforms may be deferred until started).
• If the package uses a component template: `PUT /_component_template/{name}` with the merged or new body. Then

`POST /{data_stream}/_rollover` on the rollover target so new indices pick up mappings.

• If the package uses an ingest pipeline: `PUT /_ingest/pipeline/{name}` with appended or new processors. Use the

installed package version in the versioned pipeline name.

Skip template, pipeline, and rollover when the package row has no values for them (beaconing, ded, lmd).

6. **Start transforms and create destination aliases.** Skip this block (except `.ml-anomalies-shared`) when the package has no transform pattern. Fleet's first checkpoint often skips `.all` aliases. After install, `GET /_transform/{pattern}` again if IDs were missing. Reset only if the transform is new, never started, failed, or the user asked to rebuild: `POST /_transform/{id}/_stop`, `POST /_transform/{id}/_reset`, `POST /_transform/{id}/_update` with the existing description, `POST /_transform/{id}/_start`, `POST /_transform/{id}/_schedule_now`. If al