/alibabacloud-emr-starrocks-manage

name: alibabacloud-emr-starrocks-manage
description: >
  Manage the full lifecycle of Alibaba Cloud EMR Serverless StarRocks instances — create, scale, configure, maintain and diagnose.
  Use this Skill when operations engineers, SREs, or architects need to manage StarRocks instances.
  Typical scenarios include: "create a StarRocks", "check instance status", "scale up CU", "modify configuration",
  "restart instance", "diagnose issues", etc.
  Not applicable for: writing SQL/DDL, data import/export, query tuning, materialized view configuration,
  or managing non-StarRocks products (EMR clusters, Spark, Milvus, ClickHouse, Doris, RDS, ECS).
license: MIT
compatibility: >
  Requires Alibaba Cloud CLI (aliyun >= 3.0) with AccessKey or STS Token configured.
  Verify credentials via `aliyun configure list`.
metadata:
  domain: aiops
  owner: starrocks-team
  contact: starrocks-agent@alibaba-inc.com
  required_permissions:
    - "starrocks:CreateInstanceV1"
    - "starrocks:DescribeInstances"
    - "starrocks:DescribeNodeGroups"
    - "starrocks:DescribeInstanceConfigs"
    - "starrocks:DescribeConfigHistory"
    - "starrocks:QueryUpgradableVersions"
    - "starrocks:ListGateway"
    - "vpc:DescribeVpcs"
    - "vpc:DescribeVSwitches"
    - "ecs:DescribeSecurityGroups"

Alibaba Cloud EMR Serverless StarRocks Instance Full Lifecycle Management

Manage StarRocks instances via the `aliyun` CLI. You are an SRE who understands StarRocks — you not only know how to call APIs, but also know when to call them and what parameters to use.

Authentication

Reuse the profile already configured in the `aliyun` CLI. Switch accounts with `--profile <name>`, and check configuration with `aliyun configure list`.

Domain Knowledge

Product Overview

EMR Serverless StarRocks is a fully managed service of open-source StarRocks on Alibaba Cloud, providing a high-performance, fully managed real-time analytical database service.

**Core Features**:

• **MPP Distributed Execution Framework**: Massively parallel processing to boost query performance
• **Fully Vectorized Engine**: Columnar storage and vectorized computation for efficient analytical query processing
• **Separated Storage and Compute**: Supports separated storage-compute architecture for independent scaling of storage and compute resources
• **CBO Optimizer**: Cost-based query optimizer that automatically generates optimal execution plans
• **Real-time Updatable Columnar Storage Engine**: Supports real-time data ingestion and updates
• **Intelligent Materialized Views**: Automatically maintains materialized views to accelerate query performance
• **Data Lake Analytics**: Supports querying external data sources such as OSS and MaxCompute

Use Cases

• **OLAP Multi-dimensional Analysis**: Complex multi-dimensional data analysis, ad-hoc queries, report analysis
• **Real-time Data Warehouse**: Real-time data ingestion and processing, real-time reports and dashboards, real-time risk control and analytics
• **High-concurrency Queries**: High-concurrency point queries and short queries, online analytical processing, user behavior analysis
• **Unified Analytics**: Data lake analytics (querying OSS, MaxCompute, etc.), lakehouse architecture, cross-datasource federated queries

Core Concepts

| Concept | Description | |---------|-------------| | **StarRocks Instance** | Each created StarRocks cluster (including multiple FE and multiple BE/CN nodes) is collectively called a StarRocks instance | | **CU (Compute Unit)** | Unit of compute resources; the total compute resources needed for write and query operations in StarRocks are measured in CUs | | **Compute Group** | A group of StarRocks compute nodes, containing node types such as FE, BE, and CN | | **FE (Frontend)** | Frontend node, responsible for metadata management, client connection management, query planning, and query scheduling | | **BE (Backend)** | Backend node, responsible for data storage and SQL execution (shared-nothing edition) | | **CN (Compute Node)** | Compute node, a stateless node responsible for managing hot data cache, executing data import and query computation tasks (shared-data edition) | | **Shared-nothing** | Data is stored on cloud disks or local disks; BE nodes handle both data storage and computation | | **Shared-data** | Data is persistently stored in OSS object storage; CN nodes handle computation, and local disks are used for caching |

**FE Node Roles**:

• **Leader**: Primary node, responsible for metadata writes and cluster management
• **Follower**: Secondary node, synchronizes Leader metadata, can participate in elections
• **Observer**: Observer node, only synchronizes metadata, does not participate in elections, used to scale query concurrency

Instance Types

When creating an instance, you need to choose the architecture type:

| Architecture Type | RunMode Value | Node Composition | Data Storage | Data Disk Type | Use Cases | |-------------------|---------------|------------------|--------------|----------------|-----------| | **Shared-nothing Edition** | `shared_nothing` | FE + BE | Cloud disk or local disk | ESSD cloud disk or local disk | OLAP multi-dimensional analysis, high-concurrency queries, real-time data analysis, latency-sensitive scenarios | | **Shared-data Edition** | `shared_data` | FE + CN | OSS object storage | ESSD cloud disk (cache) | Highly cost-sensitive storage with relatively lower query efficiency requirements, such as data warehouse applications |

**Shared-nothing Architecture Features**:

• BE nodes handle both data storage and computation
• Data is stored on cloud disks or local disks
• Suitable for high-performance, low-latency OLAP scenarios

**Shared-data Architecture Features**:

• Data is persistently stored in OSS object storage
• CN nodes are stateless compute nodes; local disks are primarily used for caching hot data
• Compute and storage scale independently for better cost optimization
• Table type is ident