/alibabacloud-avatar-video

name: alibabacloud-avatar-video
description: Use Alibaba Cloud DashScope API and LingMou to generate AI video and speech. Seven capabilities — (1) LivePortrait talking-head (image + audio → video, two-step), (2) EMO talking-head, (3) AA/AnimateAnyone full-body animation (three-step), (4) T2I text-to-image (Wan 2.x, default wan2.2-t2i-flash), (5) I2V image-to-video (Wan 2.x, default wan2.7-i2v-flash, supports T2I→I2V pipeline), (6) Qwen TTS (auto model/voice by scene, default qwen3-tts-vd-realtime-2026-01-15), (7) LingMou digital-human template video with random template, public-template copy, and script confirmation. Trigger when the user needs talking-head, portrait, full-body animation, text-to-image, text-to-video, or speech synthesis.
metadata:
  {
    "openclaw": {
      "emoji": "🎭",
      "requires": {
        "bins": ["ffmpeg", "ffprobe"],
        "env": [
          "DASHSCOPE_API_KEY",
          "ALIBABA_CLOUD_ACCESS_KEY_ID",
          "ALIBABA_CLOUD_ACCESS_KEY_SECRET",
          "OSS_BUCKET",
          "OSS_ENDPOINT"
        ]
      }
    }
  }

Human Avatar — Alibaba Cloud AI Video & Speech

Capabilities overview

| Capability | Script | Model / API | Region | Summary | |------|------|---------|--------|------| | **LivePortrait** | `live_portrait.py` | `liveportrait` | cn-beijing | Portrait + audio/video → talking video, two steps | | **EMO** | `portrait_animate.py` | `emo-v1` | cn-beijing | Portrait + audio → talking head, detect + generate | | **AA** (AnimateAnyone) | `animate_anyone.py` | `animate-anyone-gen2` | cn-beijing | Full-body animation: detect → motion template → video | | **T2I** | `text_to_image.py` | `wan2.x-t2i` | Multi-region | Text → image, default wan2.2-t2i-flash | | **I2V** | `image_to_video.py` | `wan2.x-i2v` | Multi-region | Image → video; T2I→I2V pipeline supported; default wan2.7-i2v-flash | | **Qwen TTS** | `qwen_tts.py` | `qwen3-tts-*` | cn-beijing / Singapore | Text → speech; auto model/voice by scene | | **LingMou** | `avatar_video.py` | LingMou SDK | cn-beijing | Template-based digital-human broadcast video |

---

Quick selection guide

Talking head (have audio/video already)     → LivePortrait
Talking head (no audio; synthesize first)   → Qwen TTS → LivePortrait
Full-body dance / motion                    → AA (AnimateAnyone)
Text → image                                → T2I (text_to_image)
Image → video                               → I2V (image_to_video)
Text → video end-to-end                     → T2I → I2V (image_to_video --t2i-prompt)
Enterprise digital human / template news    → LingMou (avatar_video)

---

Environment setup

pip install requests==2.33.1 dashscope==1.25.15 oss2==2.19.1 numpy==1.26.4
# LingMou additionally:
pip install alibabacloud-lingmou20250527==1.7.0 alibabacloud-tea-openapi==0.4.4

export DASHSCOPE_API_KEY=sk-xxxx               # Beijing-region API key
export ALIBABA_CLOUD_ACCESS_KEY_ID=xxx         # OSS upload
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=xxx
export OSS_BUCKET=your-bucket
export OSS_ENDPOINT=oss-cn-beijing.aliyuncs.com

> ⚠️ API keys for `cn-beijing` and **Singapore are not interchangeable**; use the key for the correct region. > `OSS_ENDPOINT` may include or omit the `https://` prefix; scripts normalize it.

---

1. LivePortrait — talking-head video

**When to use**: You have a portrait photo + speech and want a talking-head video quickly.

**Flow**:

Step 1: liveportrait-detect (sync)  → pass=true
  ↓
Step 2: liveportrait        (async)  → video_url

**Image**: Single person, front-facing portrait, clear face, no occlusion **Audio**: wav/mp3, < 15MB, 1s–3min **Video input**: Audio extracted automatically (ffmpeg)

# Image + audio file
python scripts/live_portrait.py \
  --image ./portrait.jpg \
  --audio ./speech.mp3 \
  --template normal --download

# Image + video (extract audio)
python scripts/live_portrait.py \
  --image ./portrait.jpg \
  --video ./speech_video.mp4 \
  --template active --download

# Public URLs
python scripts/live_portrait.py \
  --image-url "https://..." \
  --audio-url "https://..." \
  --mouth-strength 1.2 --download

**Motion templates**:

• `normal` (default, moderate motion)
• `calm` (calm; news / storytelling)
• `active` (lively; singing / hosting)

---

2. Qwen TTS — text to speech

**When to use**: Generate speech files from text (for LivePortrait, EMO, etc.).

**Default model**: `qwen3-tts-vd-realtime-2026-01-15`

Auto model selection by scene

| Scene `--scene` | Suggested model | Suggested voice | |---------------|---------|---------| | `default` / `brand` | `qwen3-tts-vd-realtime-2026-01-15` | Cherry | | `news` / `documentary` / `advertising` | `qwen3-tts-instruct-flash-realtime` | Serena / Ethan | | `audiobook` / `drama` | `qwen3-tts-instruct-flash-realtime` | Cherry / Dylan | | `customer_service` / `chatbot` / `education` | `qwen3-tts-flash-realtime` | Anna / Ethan | | `ecommerce` / `short_video` | `qwen3-tts-flash-realtime` | Cherry / Chelsie |

Available voices

| Voice | Character | |------|------| | `Cherry` | Bright, sweet female; ads / audiobooks / dubbing | | `Serena` | Mature, intellectual female; news / explainers / corporate | | `Ethan` | Steady, warm male; education / documentary / training | | `Dylan` | Expressive male; radio drama / game VO | | `Anna` | Gentle, friendly female; support / assistant / daily | | `Chelsie` | Young, fresh female; short video / e-commerce | | `Thomas` | Deep, magnetic male; brand / ads | | `Luna` | Warm, soft female; meditation / storytelling |

# Default (qwen3-tts-vd-realtime + Cherry)
python scripts/qwen_tts.py --text "Hello, welcome to Qwen TTS." --download

# Match by scene
python scripts/qwen_tts.py --text "Today's market..." --scene news --download
python scripts/qwen_tts.py --text "Once upon a time..." --scene audiobook --download

# Style via instructions
python scripts/qwen_tts.py \
  --text "Dear students..." \
  --model