/alibabacloud-bailian-voice-creator

name: alibabacloud-bailian-voice-creator
description: AI voice creation skill supporting speech recognition (ASR) and text-to-speech (TTS). Uses qwen3-asr-flash-filetrans, qwen-tts and other models. Use this skill when users need speech-to-text, text-to-speech, or audio processing. Note: on first run, it will auto-manage DashScope API Keys (create/recycle) and may auto-install the Alibaba Cloud CLI ModelStudio plugin.

AI Voice Creator

Professional-grade AI voice creation skill supporting speech recognition (ASR) and text-to-speech (TTS). Built on Alibaba Cloud DashScope API.

Security Rules (Mandatory)

1. **Never hardcode or log API Keys in plain text** in code, logs, or output files. Patterns like `api_key = "sk-..."` are strictly forbidden. 2. API Keys must only be obtained via `scripts/api_key.py`'s `get_api_key()` function, or via `os.environ.get('DASHSCOPE_API_KEY')`. 3. Generated scripts, logs, and output files must not contain any content starting with `sk-`. 4. Logs should only record operation status (success/failure), never credential information.

Execution Requirements (Mandatory)

1. **TTS must and can only use `dashscope.MultiModalConversation.call` with the `qwen-tts` model.** Using edge-tts, gTTS, ElevenLabs, Azure TTS, sambert, NLS, or any other third-party TTS service is strictly forbidden. 2. **Never generate mock/demo code or placeholder files.** Real API calls must be made; do not generate simulated scripts or blank audio files for any reason. 3. **Never auto-fallback when API calls fail.** Do not switch to other TTS services after a qwen-tts failure. Catch the exception, output a clear error message, and exit. 4. If the `dashscope` library is missing, install it first with `pip install dashscope`.

Required API Call Templates (Do Not Replace)

Standard Speech Synthesis

import dashscope
from api_key import get_api_key

api_key = get_api_key()
if api_key:
    dashscope.api_key = api_key
# If get_api_key() returns None, SDK resolves auth via environment (AK/SK, etc.)

response = dashscope.MultiModalConversation.call(
    model="qwen-tts",
    text="Text to synthesize",
    voice="Cherry"
)
audio_url = response.output.get('audio', {}).get('url', '')

Instruct-Controlled Speech Synthesis (Required when user requests a specific voice style)

response = dashscope.MultiModalConversation.call(
    model="qwen-tts",
    text="Text to synthesize",
    voice="Cherry",
    # NOTE: instructions value must be in Chinese - the qwen-tts model processes Chinese instructions
    instructions="语速快，充满热情和感染力，直播带货风格"
)

**Note: The `instructions` parameter controls voice style via natural language. Do NOT substitute it with `speech_rate`, `pitch_rate`, or `volume_rate` numeric parameters.**

Error Handling Template

import sys

try:
    response = dashscope.MultiModalConversation.call(
        model="qwen-tts", text=text, voice=voice
    )
    if response.status_code != 200:
        print(f"qwen-tts call failed: {response.code} - {response.message}")
        sys.exit(1)
except Exception as e:
    print(f"qwen-tts call failed: {e}")
    print("Please check: 1) Is DASHSCOPE_API_KEY set? 2) Is the network available?")
    sys.exit(1)
# Do NOT fallback to edge-tts, gTTS or other services here

Feature Overview

| Feature | Model | Highlights | |---------|-------|------------| | Long Audio Recognition | `qwen3-asr-flash-filetrans` | Up to 12 hours, supports emotion detection & timestamps | | Short Audio Recognition | `qwen3-asr-flash` | Up to 5 minutes, low latency | | Speech Synthesis | `qwen-tts` | Multiple voices, multilingual, instruction control | | Instruct-Controlled Synthesis | `qwen-tts` + instructions | Control voice expressiveness via natural language |

Orchestration Logic

Products and APIs

| Product | API / SDK Call | Purpose | |---------|---------------|---------| | DashScope ASR | `Transcription.async_call` + `Transcription.wait` | Long audio recognition (async) | | DashScope ASR | `POST /services/audio/asr/transcription` | Short audio recognition (sync) | | DashScope TTS | `MultiModalConversation.call` | Speech synthesis (standard / instruct-controlled) | | Alibaba Cloud CLI ModelStudio | `create-api-key` / `list-workspaces` / `delete-api-key` | API Key lifecycle management |

Decision Flow

User Request
  |
  +-- Intent: Audio -> Text (ASR)
  |     |
  |     +-- Audio duration <= 5 min AND file <= 10MB AND no emotion/timestamps needed?
  |     |     -> Short audio recognition: qwen3-asr-flash (sync, low latency)
  |     |
  |     +-- Other cases (long audio / emotion detection / timestamps needed)
  |           -> Long audio recognition: qwen3-asr-flash-filetrans (async, submit + poll)
  |
  +-- Intent: Text -> Speech (TTS)
  |     |
  |     +-- User specified voice style/emotion/speed requirements?
  |     |     -> Instruct-controlled synthesis: qwen-tts + instructions parameter
  |     |
  |     +-- Standard reading only
  |           -> Standard synthesis: qwen-tts
  |
  +-- Prerequisite: No available API Key
        -> Call api_key.py: get_api_key() auto-reads
        -> If none exists: generate_api_key() creates via Alibaba Cloud CLI and saves

Call Sequence

**Speech Recognition (Long Audio)**: 1. `get_api_key()` -> Get DashScope API Key 2. `Transcription.async_call(model, file_urls, language_hints)` -> Submit async task, get task_id 3. `Transcription.wait(task=task_id)` -> Poll until task completes 4. Get recognition result JSON from `output.results[].transcription_url` 5. Parse `transcripts[].text` / `sentences[]` / `emotion` from JSON

**Speech Recognition (Short Audio)**: 1. `get_api_key()` -> Get DashScope API Key 2. `POST /services/audio/asr/transcription` -> Sync call, returns recognized text directly

**Speech Synthesis (Standard / Instruct-Controlled)**: 1. `get_api_key()` -> Get DashScope API Key 2. `MultiModalConversation.call(model, text, voice, [instructi