/alibabacloud-emr-cluster-manage

name: alibabacloud-emr-cluster-manage
description: >
  Manage the full lifecycle of Alibaba Cloud E-MapReduce (EMR) ECS clusters—creation, scaling, renewal, and status queries.
  Use this Skill when users want to set up big data clusters, view cluster status, add nodes, release nodes, configure auto-scaling,
  check cluster and node states, or diagnose creation failures.
  Also applicable for scenarios like "create a Hadoop cluster", "data lake cluster", "running out of resources",
  "check my cluster", "renew", etc.
  NOTE: This Skill does NOT support cluster deletion, release, or termination under any circumstances.
  Any request to delete or terminate a cluster will be refused and redirected to the EMR console.
license: MIT
compatibility: >
  Requires Alibaba Cloud CLI (aliyun >= 3.3.3), with AccessKey or STS Token configured.
  Verify credentials via `aliyun configure list`.
metadata:
  domain: aiops
  owner: emr-team
  contact: yanhui.jy@alibaba-inc.com
  required_permissions: references/ram-policies.md

Alibaba Cloud EMR Cluster Full Lifecycle Management

Manage EMR clusters via `aliyun` CLI. You are an EMR-savvy SRE—not just an API caller, but someone who knows when to call APIs and what parameters to use.

Authentication

Reuse the configured `aliyun` CLI profile. Switch accounts with `--profile <name>`, check configuration with `aliyun configure list`.

Before execution, read [ram-policies.md](references/ram-policies.md) if you need to confirm the minimum RAM authorization scope.

Installation

**Pre-check: Aliyun CLI >= 3.3.3 required** > Run `aliyun version` to verify >= 3.3.3. If not installed or version too low, > run `curl -fsSL https://aliyuncli.alicdn.com/setup.sh | bash` to update, > or see `references/cli-installation-guide.md` for installation instructions.

**Pre-check: Aliyun CLI plugin update required** > [MUST] run `aliyun configure set --auto-plugin-install true` to enable automatic plugin installation. > [MUST] run `aliyun plugin update` to ensure that any existing plugins are always up-to-date.

**[MUST] AI-Mode Setup** — Before executing any CLI commands, enable AI-Mode and set User-Agent:

aliyun configure ai-mode enable
aliyun configure ai-mode set-user-agent --user-agent "AlibabaCloud-Agent-Skills/alibabacloud-emr-cluster-manage"

After all workflow operations are complete, disable AI-Mode:

aliyun configure ai-mode disable

**[MUST] CLI User-Agent** — Every `aliyun` CLI command invocation must include: `--user-agent AlibabaCloud-Agent-Skills/alibabacloud-emr-cluster-manage`

Execution Principles

1. **Check documentation before acting**: Before calling any API, consult `references/api-reference.md` to confirm parameter names and formats. Never guess parameter names from memory. 2. **Return to documentation on errors — MANDATORY**: When any API call fails, STOP. Do NOT retry with variations. Go directly to `references/api-reference.md` and `references/error-recovery.md`, find the exact error code, read the correct parameter specification, then retry ONCE with the corrected command. Blind retry loops are prohibited. 3. **No intent downgrade**: If user requests "create", you must create—no substituting with "find existing". 4. **Verify before executing**: Before running RunCluster or CreateCluster, cross-check your constructed command against the canonical example in `references/getting-started.md`. Confirm every field name matches exactly.

EMR Domain Knowledge

For detailed explanations of cluster types, deployment modes, node roles, storage-compute architecture, recommended configurations, and payment methods, refer to [Cluster Planning Guide](references/cluster-lifecycle.md#一规划阶段).

Key decision quick reference:

• **Cluster Type**: 80% of scenarios choose DATALAKE; real-time analytics choose OLAP; stream processing choose DATAFLOW; NoSQL choose DATASERVING
• **Deployment Mode**: Production uses HA (3 MASTER), dev/test uses NORMAL (1 MASTER); HA mode **must select ZOOKEEPER** (required for master standby switching), and Hive Metastore must use external RDS
• **Node Roles**: MASTER runs management services; CORE stores data (HDFS) + compute; TASK is pure compute without data (preferred for elasticity, can use Spot); GATEWAY is job submission node (avoid submitting directly on MASTER); MASTER-EXTEND shares MASTER load (only HA clusters support)
• **Storage-Compute Architecture**: Recommended storage-compute separation (OSS-HDFS), better elasticity, lower cost; before choosing storage-compute separation, must enable HDFS service for target Bucket in OSS console; choose storage-compute integrated (HDFS + d-series local disks) when extremely latency-sensitive
• **Payment Method**: Dev/test uses PayAsYouGo, production uses Subscription
• **Component Mutual Exclusion**: SPARK2/SPARK3 choose one; HDFS/OSS-HDFS choose one; STARROCKS2/STARROCKS3 choose one

Create Cluster Workflow

When creating a cluster, must interact with user in the following steps, **cannot skip any confirmation环节**:

1. **Confirm Region**: Ask user for target RegionId (e.g., cn-hangzhou, cn-beijing, cn-shanghai) 2. **Confirm Purpose**: Dev/test / small production / large production, determines deployment mode (NORMAL/HA) and payment method 3. **Confirm Cluster Type and Application Components**:

• First recommend cluster type based on user needs (DATALAKE/OLAP/DATAFLOW/DATASERVING/CUSTOM)
• Then show available component list for that type (refer to cluster type table above), let user select components to install
• If user is unsure, give recommended combination (e.g., DATALAKE recommends HADOOP-COMMON + HDFS + YARN + HIVE + SPARK3)
• Clearly inform user of component mutual exclusion rules and dependencies

4. **Confirm Hive Metadata Storage** (must ask when HIVE is selected):

• **local**: Use MASTER local MySQL to store metadata, simple no configuration, suitable for dev/test
• **External RDS**: Use independent RDS MySQL instance, me