/alibabacloud-video-translation

name: alibabacloud-video-translation
description: |
  Alibaba Cloud IMS (Intelligent Media Services) based video translation Skill. Supports subtitle extraction (ASR/OCR), translation, and speech synthesis translation modes.
  Trigger words: "视频翻译", "translate video", "翻译视频", "字幕翻译", "video translation"

Video Translation Skill

One-click video translation powered by Alibaba Cloud IMS, supporting subtitle-level and speech-level translation.

---

Input Format Requirements

> **IMPORTANT**: Different APIs use different address formats!

API Address Format Reference

| API | Address Format | Example | |-----|----------------|---------| | `SubmitIProductionJob` (subtitle extraction) | **`oss://` format** | `oss://my-bucket/videos/test.mp4` | | `SubmitVideoTranslationJob` (video translation) | **HTTP URL format** | `https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4` |

> **Key**: Subtitle extraction uses `oss://`, video translation uses HTTP URL!

User Input Handling

| User Input Type | Processing Method | |-----------------|-------------------| | HTTP URL format | Use directly for video translation; convert to `oss://` if subtitle extraction needed | | `oss://` format | Use directly for subtitle extraction; convert to HTTP URL for video translation | | Local video | **MUST ask** for OSS upload path, save both formats after upload |

Format Conversion Rules

oss:// format ⇄ HTTP URL format

oss://my-bucket/videos/test.mp4
    ⇄
https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4

**Conversion Formula**:

• `oss://<bucket>/<path>` → `https://<bucket>.oss-<region>.aliyuncs.com/<path>`
• HTTP URL does not require signing, use Bucket domain format directly

Local Video Processing Flow

User provides local video path
    │
    ├─ AskUserQuestion: "Please provide OSS upload path (format: oss://<bucket>/<path>/<filename>.mp4)"
    │
    ├─ User specifies upload path
    │   ├─ Check if Bucket exists
    │   ├─ Upload file: aliyun oss cp <local_path> <oss_path>
    │   ├─ Save oss:// format → for subtitle extraction
    │   └─ Save HTTP URL format → for video translation
    │
    └─ User does not specify path → STOP, user MUST provide upload path

**Upload Command**:

aliyun oss cp <local_path> oss://<bucket>/<path>/<filename>.mp4

**Save both formats after upload**:

Local: /Users/demo/videos/test.mp4
Uploaded to: oss://my-bucket/videos/test.mp4
    ├─ oss:// format: oss://my-bucket/videos/test.mp4 (for subtitle extraction)
    └─ HTTP URL: https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4 (for video translation)

---

Execution Gate Checklist

> **Strict Requirement**: Agent MUST execute in phase order, cannot proceed without passing current phase!

Phase 0: Environment and Credential Check (HARD-GATE)

| Check Item | Command | Pass Condition | Failure Handling | |------------|---------|----------------|------------------| | CLI version | `aliyun version` | >= 3.3.1 | STOP, see [cli-installation-guide.md](references/cli-installation-guide.md) | | Credential status | `aliyun configure list` | Valid status | STOP, guide configuration | | Plugin installation | `aliyun configure set --auto-plugin-install true` | Set | Auto-set |

> **HARD-GATE**: Cannot proceed with any subsequent operations without passing!

---

Phase 1: Translation Mode Confirmation (BLOCKING)

AskUserQuestion: "Do you need subtitle translation (translate subtitles only) or speech translation (translate subtitles + replace voiceover)?"

┌─ Subtitle translation → NeedSpeechTranslate: false
└─ Speech translation → NeedSpeechTranslate: true

⚠️ No reply received → STOP, cannot proceed!

> **DO NOT infer translation mode from input type!**

---

Phase 2: Subtitle Processing Confirmation (BLOCKING)

AskUserQuestion: "Do you need to erase original subtitles from the video? Do you need to burn-in translated subtitles?"

⚠️ No reply received → STOP, cannot proceed!

**Parameter Mapping**:

| Feature | Parameter | Value | |---------|-----------|-------| | Erase original subtitles | `DetextArea` | `"Auto"` / coordinates / not set (no erasure) | | Burn-in new subtitles | `SubtitleConfig` | config object / not set (no burn-in) |

---

Phase 3: Output Path Confirmation (Non-blocking)

| Condition | Processing Method | |-----------|-------------------| | User explicitly specifies | Use user's path | | User does not specify | Use default path and inform user |

**Default Output Rules**:

• Bucket: Same bucket as input video
• Directory: Same directory as input video
• Filename: `{source}_translated_{random8}.mp4`
• Example: `oss://bucket/videos/demo.mp4` → `oss://bucket/videos/demo_translated_a1b2c3d4.mp4`

> DO NOT use shell variables, use Python: `python3 -c "import random; print(''.join(random.choices('abcdefghijkmnpqrstuvwxyz23456789', k=8)))"`

---

Phase 4: Subtitle Review Confirmation (Conditional Blocking)

| Trigger Condition | Processing Method | |-------------------|-------------------| | User chooses to review subtitles | BLOCKING, MUST wait for user confirmation of review result | | User does not need review | Non-blocking, proceed |

> **CRITICAL**: After subtitle extraction, MUST output content as-is for user review, DO NOT change format!

---

Scenario Entry Selector

> **Key Points**: > 1. When user inputs local video, MUST first upload to OSS and get HTTP URL > 2. When user does not provide subtitle, MUST ask if subtitle extraction and review is needed

User inputs video
    │
    ├─ Local video?
    │   └─ Yes → AskUserQuestion: "Please provide OSS upload path"
    │       ├─ User provides path → Upload to OSS → Convert to HTTP URL → Continue
    │       └─ User does not provide → STOP
    │
    ├─ oss:// format?
    │   └─ Yes → Inform user to convert to HTTP URL format
    │
    └─ HTTP URL format? → Continue
        │
        ├─ User provides SRT file?
        │   ├─ Yes → Input ty