/alibabacloud-pai-dlc-job-diagnostics

name: alibabacloud-pai-dlc-job-diagnostics
description: |
  PAI-DLC job diagnostics and health inspection. Queuing-stuck root cause
  analysis, failed-job localization, cluster health checks.
  Companion to the `alibabacloud-pai-dlc-job` skill (read-only — no writes).
  Triggers: "diagnose", "diagnose job", "job stuck", "why queuing",
  "queue stuck", "stuck in queue", "job failed", "failure reason",
  "healthcheck", "health check", "inspect job", "inspection".

PAI-DLC Job Diagnostics and Health Inspection

Read-only diagnostic analysis for PAI-DLC distributed training jobs, covering three scenarios:

• **Queuing-stuck root cause analysis** — quota check, node scheduling

diagnosis, hyper-node availability

• **Failed-job localization** — failure classification, logs/events evidence

chain, root cause identification

• **Cluster health inspection** — training throughput, hang detection,

SanityCheck, restart stability

**Architecture**: PAI-DLC Job (read-only queries) + PAI Studio Resource Diagnosis API (queuing scenario).

0. Dependencies

This skill performs **read-only diagnostics** only. All write operations (create / update / stop jobs, resource discovery, etc.) live in the companion skill `alibabacloud-pai-dlc-job`. The two skills are complementary in responsibility and share a common field contract.

| Prerequisite Skill | Role | When to switch to it | |--------------------|------|----------------------| | `alibabacloud-pai-dlc-job` | Write ops (create/update/stop) + AIWorkSpace resource discovery | Creating / modifying / stopping jobs, or discovering Image / Dataset / CodeSource | | This skill | Read-only diagnostics (logs / events / sanity-check / queuing root cause) | Job already exists — troubleshooting or health inspection |

**Discover and install the prerequisite skill:**

# Discover available skills
npx skills add aliyun/alibabacloud-aiops-skills --skill alibabacloud-find-skills
# Install the alibabacloud-pai-dlc-job skill itself
npx skills add aliyun/alibabacloud-aiops-skills --skill alibabacloud-pai-dlc-job

**Cross-skill field contract:** The `--job-id` / `--pod-id` values this skill consumes are produced verbatim by `alibabacloud-pai-dlc-job` via `list-jobs` / `get-job --cli-query "Pods[0].PodId"` — no transformation needed. `--region` / `--workspace-id` follow the same resolution rules in both skills.

Installation Requirements

> **Pre-check: Aliyun CLI >= 3.3.1 required** > Run `aliyun version` to verify >= 3.3.1. If not installed or version too low, > see [references/cli-installation-guide.md](references/cli-installation-guide.md). > Then [MUST] run `aliyun configure set --auto-plugin-install true`.

> **Note on `--user-agent`:** Every API-invoking `aliyun` command in this skill MUST > include `--user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job-diagnostics`. Client-side helpers > (`aliyun version`, `aliyun configure ...`, `aliyun plugin ...`, > `aliyun <product> --help`) do not invoke remote APIs and therefore do not require > the flag.

aliyun version
aliyun configure set --auto-plugin-install true
aliyun plugin update
aliyun pai-dlc --help
aliyun paistudio --help >/dev/null 2>&1 || aliyun plugin install --names aliyun-cli-paistudio

aliyun configure ai-mode enable
aliyun configure ai-mode set-user-agent --user-agent "AlibabaCloud-Agent-Skills/alibabacloud-pai-dlc-job-diagnostics"
# After session: aliyun configure ai-mode disable

Authentication

> **Pre-check: Alibaba Cloud Credentials Required** > > **Security Rules:** > - **NEVER** read, echo, or print AK/SK values > - **NEVER** ask the user to input AK/SK directly > - **ONLY** use `aliyun configure list` to check credential status > > ```bash > aliyun configure list > ``` > Check the output for a valid profile (AK, STS, or OAuth identity). > > **If no valid profile exists, STOP here.** > 1. Obtain credentials from [Alibaba Cloud Console](https://ram.console.aliyun.com/manage/ak) > 2. Configure credentials **outside of this session** > 3. Return and re-run after `aliyun configure list` shows a valid profile

RAM Permissions

> **[MUST] Permission Failure Handling:** When any command fails due to permission errors: > 1. Read `references/ram-policies.md` for the full permission list > 2. Use `ram-permission-diagnose` skill to guide the user > 3. Pause and wait until the user confirms permissions have been granted

| Product | Permissions | Purpose | |---------|-------------|---------| | pai-dlc | `pai:GetJob`, `pai:GetPodLogs`, `pai:GetJobEvents`, `pai:GetPodEvents`, `pai:ListJobSanityCheckResults` | Job information collection | | paistudio | `paistudio:GetQuotaWorkloadDiagnosis` | Queuing resource diagnosis |

Parameter Confirmation

> **IMPORTANT: Parameter Confirmation** — Before executing any command, > ALL user-customizable parameters (RegionId, JobId, etc.) MUST be confirmed with the user.

| Parameter | Required | Description | |-----------|----------|-------------| | `region` | Yes | Region where the job runs | | `job_id` | Yes | DLC job ID (e.g., `dlcXXX`) |

---

Entry Routing

When a diagnostic request arrives, first call `get-job` to fetch job status, then route by status:

| Job status | Route to scenario | |------------|-------------------| | `Queuing` / `Creating` | → Queuing-stuck root cause analysis | | `Failed` | → Failed-job localization | | `Running` | → Health inspection | | `Stopped` | Inform the user "job was actively stopped", no diagnosis | | `Succeeded` | → Historical review (follow Scenario 3 Execution steps) |

**Edge case — job was queuing but is now Stopped/Succeeded**: If the user describes the job as "stuck in queue" but `get-job` shows `Stopped` or `Succeeded`, still route to Scenario 1 (queuing analysis) but expect the resource diagnosis API to return HTTP 400. Follow the "Fallback on API Failure" procedure in Scenario 1.

Users may also directly request a specific scenario (e.g., "run a health inspection" even wh