/byted-util-volcengine-detect-retry

name: byted-util-volcengine-detect-retry
description: Orchestrates Volcengine Cloud Detect (云拨测) false-alarm reduction for failing periodic-task dial points. Use for scheduled task scans, retrying failed nodes, control-testing node health, distinguishing real target outages from broken dial points, and suggesting same-type LastMile/IDC/Private replacement nodes. Runs the byted-util-volcengine-cloud-detect volc_detect.py CLI only; requires that skill and ships no API/script implementation.

Volcengine Detect Retry

Reduce Volcengine Cloud Detect (云拨测) operations toil. This skill confirms whether a failing dial point is a real outage or just noise, and — when a dial point itself is broken — suggests a similar replacement node.

**This skill ships no script of its own.** It is pure orchestration: every action is one of the `byted-util-volcengine-cloud-detect` skill's `volc_detect.py` CLI commands (`get-task`, `get-result`, `list-nodes`, `fuzzy-search-nodes`, `create-task`, `delete-task`). Follow the workflow below by invoking those commands in sequence and applying the decision logic to their JSON output.

**Requires**: `byted-util-volcengine-cloud-detect` skill. Do not reimplement any Cloud Detect API call here.

Locate The Base CLI

Resolve `volc_detect.py` dynamically (never hard-code an absolute path):

VOLC_DETECT="${VOLC_DETECT:-$(find "${SKILLS_DIR:-$HOME/Code/Skill}" -type f \
  -path '*/byted-util-volcengine-cloud-detect/scripts/volc_detect.py' 2>/dev/null | head -n 1)}"
test -n "$VOLC_DETECT" || { echo "volc_detect.py not found"; exit 1; }
alias vd="python3 \"$VOLC_DETECT\""

All commands below are written as `python3 "$VOLC_DETECT" <command>`.

Prerequisites

Volcengine credentials must be present (the base skill enforces this). If either is missing, follow the base skill's Prerequisites section to collect and persist them:

test -n "$VOLC_ACCESSKEY" && test -n "$VOLC_SECRETKEY"

Decision Logic

For each dial point whose **latest** sample in the scan window failed:

1. **Retry 5 times** (configurable) against the **same** monitored target via a temporary `create-task`.

• **All retries succeed** → transient glitch → **ignore** (`transient_issue`).
• **Any retry fails** → go to control validation.

2. **Control validation**: a temporary `create-task` dials a trusted domain from the **same** node.

• China nodes → `https://cn.bing.com`
• Overseas nodes → `https://www.google.com`
• **Control succeeds** → `real_failure` (target is genuinely down for that node) → **ALERT**.
• **Control fails** → `node_issue` (the dial point itself is broken) → **find similar

replacement nodes and suggest replacing it**. 3. Temporary retry/control tasks are **always** deleted (`delete-task`).

By default only **LastMile** dial points (`client_info.type == "1"`, node name suffix `（LM）`) are auto-validated, because they are the noisy ones. Include IDC/Private only on request.

Replacement-node priority (`node_issue` only): 1. **Mandatory**: same node type (LastMile / IDC / Private). 2. **Preferred**: same province, then same country.

Success Rule

A result sample is **success** when `basic_detail.usability_info.status` is `"success"` (or boolean `true`); it is a **failure** when status is `"fail"` (the `usability_info.reason` array, e.g. `["主机未找到(601)"]`, and `basic_detail.error_msg` explain why). If no `usability_info` is present, fall back to `http_detail.http_code` in `200–499` = reachable.

Workflow: Scan One Periodic Task

Step 1 — Fetch the task and its recent samples

# Task must be Running (status == 2) to be actively monitoring.
python3 "$VOLC_DETECT" get-task --task-id <TASK_ID>

# Pull the last hour of results (API limits the range to <= 1 hour, PageSize <= 500).
NOW=$(date +%s); START=$((NOW-3600))
python3 "$VOLC_DETECT" get-result --task-id <TASK_ID> \
  --start-time $START --end-time $NOW --page-size 500

Step 2 — Identify failing nodes (latest sample per node)

From the `Data` array, group samples by node and keep the **latest** (max `basic_detail.timestamp`) per node. A node is identified by its `basic_detail.client_info` (`region` / `city` / `isp` / `type`). Map that back to a node `id` + full name with `list-nodes` (match `region`, `city`, `isp` substrings; if `client_info.type` is present, also match node type to disambiguate LM vs IDC).

python3 "$VOLC_DETECT" list-nodes

Keep only nodes whose latest sample is a **failure**. By default, drop non-LastMile nodes (`client_info.type != "1"`) unless the user asked to include IDC/Private. If a `client_info` maps to multiple nodes (ambiguous) or none, mark it `inconclusive` and do not auto-page.

Step 3 — Retry each failing node 5× against the same target

`get-task` gives the monitored `address`. Create a temporary task on the failing node only:

ADDRESS="<the task's address>"; LINE_ID=<failing node id>
NOW=$(date +%s); FINISH=$((NOW + 5*60 + 120))   # 5 rounds @ 60s + margin
python3 "$VOLC_DETECT" create-task \
  --address "$ADDRESS" --name "retry_${LINE_ID}_${NOW}" \
  --type 1 --node-count 1 --interval-seconds 60 \
  --finish-time $FINISH --line-ids $LINE_ID --http-method 1

Wait for ~5 samples (poll `get-task` until `status == 6`, or sleep `rounds × interval`), fetch the results, then **delete** the temporary task:

END=$(date +%s); START=$((END-3600))
python3 "$VOLC_DETECT" get-result --task-id <RETRY_TASK_ID> \
  --start-time $START --end-time $END --page-size 500
python3 "$VOLC_DETECT" delete-task --task-id <RETRY_TASK_ID>

• **All retry samples succeed** → `transient_issue` → ignore. Done for this node.
• **Any retry fails** → continue to Step 4.

Step 4 — Control validation from the same node

Pick the control domain by node locality (node name starting with `中国` → CN):

# China node:   CONTROL="https://cn.bing.com"
# Overseas node: CONTROL="https://www.google.com"
NOW=$(date +%s); FINISH=