/alibabacloud-dataworks-datastudio-develop

name: alibabacloud-dataworks-datastudio-develop
description: |
  DataWorks data development Skill. Create, configure, validate, deploy, update, move, and rename nodes and workflows.
  Manage components, file resources, and UDF functions. Covers 150+ node types: Shell, SQL, Python, DI, Flink, EMR, etc.
  Supports scheduled and manual workflow orchestration via aliyun CLI or Python SDK.
  WARNING: Supports mutating operations (Move, Rename) requiring explicit user confirmation. Delete operations are NOT supported by this skill.
  Triggers: DataWorks, data development nodes, workflows, FlowSpec, scheduling tasks, data integration, ETL pipelines, .spec.json.
  Also triggers for Alibaba Cloud data development, scheduling node configuration, FlowSpec format, or DI task orchestration.

DataWorks Data Development

❗ 5-SECOND SUMMARY — Read This First

> **Credentials**: Run `aliyun configure list` first. The CLI is almost always pre-configured (STS token). Do NOT search for credential files — just check the CLI config. > > **Install plugin first**: Run `aliyun plugin install --names dataworks-public`. All commands use plugin mode (kebab-case): `aliyun dataworks-public create-node ...` > > **APIs**: `create-workflow-definition` → `create-node` (per node) → `create-pipeline-run` (deploy). For updates: `update-node` → `create-pipeline-run`. **NEVER use** `deploy-file`, `submit-file`, `create-file`, `create-business`. > > **FlowSpec**: Copy the exact JSON from the "Quick Start" section below. Do NOT guess the format. `version` is `"2.0.0"`, `kind` is `"CycleWorkflow"` or `"Node"`. Common wrong values: `apiVersion`, `type`, `Workflow`, `metadata` — all wrong. > > **Publish after update**: `update-node` → `create-pipeline-run(type=Online)` → poll `get-pipeline-run` → `exec-pipeline-run-stage`. `deploy-file` does NOT work.

⚡ MANDATORY: Read Before Any API Call

**These absolute rules are NOT optional — violating ANY ONE means the task WILL FAIL:**

**Pre-check: Aliyun CLI >= 3.3.3 required** > Run `aliyun version` to verify >= 3.3.3. If not installed or version too low, > run `curl -fsSL --connect-timeout 10 --max-time 120 https://aliyuncli.alicdn.com/setup.sh | bash` to update, > or see `references/cli-installation-guide.md` for installation instructions.

**Aliyun CLI Setup (execute before first use)**:

# Plugin install (required)
aliyun plugin install --names dataworks-public
# Plugin update (run periodically)
aliyun plugin update --names dataworks-public
# AI-Mode: available commands are enable/disable/set-user-agent
# This skill DISABLES AI-Mode (exact parameter control required)
aliyun configure ai-mode disable
# Do NOT run: aliyun configure ai-mode enable
# Set user-agent for tracking
aliyun configure ai-mode set-user-agent AlibabaCloud-Agent-Skills/alibabacloud-dataworks-datastudio-develop

Per-command UA flag (business commands only): every `aliyun dataworks-public` invocation must append `--user-agent AlibabaCloud-Agent-Skills/alibabacloud-dataworks-datastudio-develop`.

0. **FIRST THING: Check CLI credentials.** Before ANY `aliyun` command, run `aliyun configure list`. The CLI is typically pre-configured with valid STS token credentials — **do NOT search for credential files** (e.g., testconfig.json) before checking. If `aliyun configure list` shows a valid profile, use it directly. If multiple profiles exist, run `aliyun configure switch --profile <name>` to select the correct one. Priority: prefer a profile whose name contains `dataworks` (case-insensitive); otherwise use `default`. **Do NOT skip this step. Do NOT run any `aliyun dataworks-public` command before switching.** NEVER read/echo/print AK/SK values. 1. **Install plugin before first use.** Run `aliyun plugin install --names dataworks-public`. If already installed, run `aliyun plugin update --names dataworks-public` to ensure latest version. The plugin provides kebab-case commands (`create-node`, `create-workflow-definition`, etc.) which are the required invocation form. 2. **ONLY use plugin mode (kebab-case).** Every DataWorks API call must look like: `aliyun dataworks-public create-node --project-id ... --spec '...'`. Never use PascalCase RPC (`CreateNode`, `CreateWorkflowDefinition`) — always use plugin mode. 3. **ONLY use these commands for create:** `create-workflow-definition` → `create-node` (per node, with `--container-id`) → `create-pipeline-run` (to deploy). 4. **ONLY use these commands for update:** `update-node` (incremental, `kind:Node`) → `create-pipeline-run` (to deploy). Never use `import-workflow-definition`, `deploy-file`, or `submit-file` for updates or publishing. 4a. **ONLY use these commands for deploy/publish:** `create-pipeline-run --type Online --object-ids <ID>` → `get-pipeline-run --id <PipelineRunId>` (poll) → `exec-pipeline-run-stage --id <PipelineRunId> --code <StageCode>` (advance). **NEVER use** `deploy-file`, `submit-file`, `list-deployment-packages`, or `get-deployment-package` — these are all legacy APIs that will fail. ⚠️ `--object-ids` is **space-separated bare IDs** (e.g. `--object-ids 7567482277219412494`), NOT a JSON array string. Wrapping it as `'["ID"]'` will produce `未找到发布对象: [["ID"]]` because the CLI passes the literal bracket text as the ID.

5. **If `create-workflow-definition` or `create-node` returns an error, FIX THE SPEC — do NOT fall back to legacy APIs.** Error 58014884415 / `0x5083000000000005` ("Spec JSON parse failed") means your FlowSpec JSON format is wrong (e.g., used `"kind":"Workflow"` instead of `"kind":"CycleWorkflow"`, or `"apiVersion"` instead of `"version"`, or used a flat `{"type":"SHELL","content":"..."}` structure instead of the `{"version":"2.0.0","kind":"Node","spec":{"nodes":[...]}}` structure). **Stop guessing and copy the exact Spec from the Quick Start below, then modify only the values you need.** 6. **Run CLI commands directly — do NOT create wrapper scripts.** Never create `.sh` scripts to batch API calls. Run each `aliyun` co