/alibabacloud-dlf-manage

name: alibabacloud-dlf-manage
description: |
  Query Catalog, database, and table metadata resources in Alibaba Cloud Data Lake Formation (DLF).
  Provides read-only queries via the DLF OpenAPI Python SDK, supporting listing and viewing
  Catalogs, databases, tables with their detailed information and Schema definitions.
  Use cases: "list available Catalogs", "list databases", "view table schema",
  "search tables", "search tables by name", "fuzzy search", "view DLF metadata",
  "what databases are in the data lake", "what columns does a table have",
  "find tables whose name contains xxx".
  This Skill only contains read-only operations — no create, modify, or delete operations.

DLF Data Lake Metadata Query

Query Catalog, Database, and Table metadata resources in Alibaba Cloud Data Lake Formation (DLF).

> **CRITICAL: Use only the Python SDK script provided by this Skill.** > All operations go through the DLF Python SDK (`alibabacloud-dlfnext20250310`) via `scripts/dlf_metadata_query.py`. > This Skill does not invoke any shell-based command-line client and does not require AI-Mode configuration. > > - **DO NOT** attempt access via any shell-based command-line client — DLF is not exposed through one in this Skill > - **DO NOT** use curl, wget, or other HTTP clients to call the DLF API directly > - **MUST** use the `scripts/dlf_metadata_query.py` script provided by this Skill, which wraps the DLF Python SDK > - All query operations are executed via `python3 scripts/dlf_metadata_query.py <action> [options]`

Architecture

Catalog (Data Catalog)
  └── Database
        └── Table
              ├── Schema (column definitions)
              ├── PartitionKeys (partition keys)
              ├── PrimaryKeys (primary keys)
              └── Options (table properties)

Installation

pip install -r requirements.txt

`requirements.txt` pins the full transitive dependency closure (including `alibabacloud-dlfnext20250310==3.0.0`) for reproducible installs.

> **Pre-check: Python SDK dependency** > > ```bash > python3 -c "from alibabacloud_dlfnext20250310.client import Client; print('SDK OK')" > ``` > If not installed, run `pip install -r requirements.txt`.

Authentication

> **Pre-check: Alibaba Cloud Credentials Required** > > Use the default credential chain (CredentialClient) to obtain credentials automatically. Supported sources (in priority order): > 1. Environment variables (ALIBABA_CLOUD_ACCESS_KEY_ID / ALIBABA_CLOUD_ACCESS_KEY_SECRET) > 2. Configuration file (~/.alibabacloud/credentials) > 3. ECS Instance RAM Role > 4. OIDC Role ARN > > **Security Rules:** > - **NEVER** read, echo, or print AK/SK values > - **NEVER** ask the user to input AK/SK directly in the conversation or command line > - **NEVER** explicitly handle or pass AK/SK in code — rely on the default credential chain > > See https://help.aliyun.com/document_detail/378659.html for credential configuration details.

RAM Permissions

This Skill only involves read-only operations (List / Get). See [references/ram-policies.md](references/ram-policies.md) for the full permission list.

> **[MUST] Permission Failure Handling:** When any command or API call fails due to permission errors at any point during execution, follow this process: > 1. Read `references/ram-policies.md` to get the full list of permissions required by this SKILL > 2. Pause and wait until the user confirms that the required permissions have been granted

Parameter Confirmation

> **IMPORTANT: Parameter Confirmation** — Before invoking the API, > the following user-specific parameters must be confirmed with the user; do not assume them. > Region defaults to cn-hangzhou; if the user does not specify one, use the default without asking.

| Parameter | Required | Description | Default | |------|------|------|--------| | `region` | No | Region ID | cn-hangzhou | | `catalog_name` | Conditional | Catalog name (`--catalog`, required for GetCatalog) | - | | `catalog_id` | Conditional | Catalog ID (`--catalog-id`, required when querying databases/tables, e.g. clg-paimon-xxxx) | - | | `database` | Conditional | Database name (`--database`) | - | | `table` | Conditional | Table name (`--table`) | - |

Core Workflow

> The script automatically reads AK/SK from environment variables and reports a clear error if they are missing. > Region defaults to cn-hangzhou; use the default if the user does not specify one.

**You MUST use** `scripts/dlf_metadata_query.py` to query metadata. Do not use shell-based command-line clients or curl. Actions are in **kebab-case**.

> **CRITICAL — list vs. list-*-details: pick the lightest action that satisfies the request.** > - For listing names / IDs (including fuzzy search): use `list-databases` / `list-tables`. These call the `ListDatabases` / `ListTables` API. > - For full attributes / Schema / properties: use `list-database-details` / `list-table-details` / `get-database` / `get-table`. These call the heavier `*-details` / `Get*` APIs. > - **Default to the lightweight `list-*` action** unless the user explicitly asks for full configuration, Schema, or properties. Calling `list-*-details` when only names are needed is incorrect.

Query Operations

# ---- Catalog ----

# 1. List all Catalogs (names + minimal info — preferred for listing/searching)
python3 scripts/dlf_metadata_query.py list-catalogs

# 2. Fuzzy-search Catalogs by name (uses ListCatalogs)
python3 scripts/dlf_metadata_query.py list-catalogs --pattern test

# 3. Get Catalog details (by name) — use only when full Catalog config is needed
python3 scripts/dlf_metadata_query.py get-catalog --catalog <catalog_name>

# 4. Get Catalog details (by ID) — use only when full Catalog config is needed
python3 scripts/dlf_metadata_query.py get-catalog-by-id --id <catalog_id>

# ---- Database ----

# 5. List databases (NAMES only — DEFAULT for "list / show / which databases", calls ListDatabases)
python3 scripts/dlf_metadata_query.py list