/ai-provider-elevenlabs

name: ai-provider-elevenlabs
description: ElevenLabs voice AI SDK patterns for TypeScript/Node.js -- text-to-speech, streaming, voice cloning, speech-to-speech, pronunciation control, and conversational AI

ElevenLabs Patterns

> **Quick Guide:** Use the official `@elevenlabs/elevenlabs-js` package to interact with the ElevenLabs API. Use `client.textToSpeech.convert()` for full audio generation or `client.textToSpeech.stream()` for low-latency streaming. Voice settings (`stability`, `similarityBoost`, `style`) control output character. Use `eleven_v3` for best quality, `eleven_flash_v2_5` for lowest latency, or `eleven_multilingual_v2` for stable long-form content. The SDK returns `ReadableStream<Uint8Array>` -- pipe to files or HTTP responses. Use `@elevenlabs/client` for real-time conversational AI agents.

---

<critical_requirements>

CRITICAL: Before Using This Skill

**(You MUST use `@elevenlabs/elevenlabs-js` for server-side TTS, voice management, and speech-to-speech -- use `@elevenlabs/client` only for conversational AI agents)**

**(You MUST never hardcode API keys -- always use environment variables via `process.env.ELEVENLABS_API_KEY` which the SDK reads automatically)**

**(You MUST consume the `ReadableStream<Uint8Array>` returned by `convert()` and `stream()` -- unconsumed streams leak resources)**

**(You MUST choose the correct model for your use case -- `eleven_v3` for quality, `eleven_flash_v2_5` for speed, `eleven_multilingual_v2` for long-form stability)**

**(You MUST pass `voiceId` as the first positional argument to all `textToSpeech` methods -- it is NOT inside the options object)**

</critical_requirements>

---

**Auto-detection:** ElevenLabs, elevenlabs, ElevenLabsClient, textToSpeech.convert, textToSpeech.stream, eleven_multilingual_v2, eleven_flash_v2_5, eleven_v3, speechToSpeech, voices.search, voice cloning, ELEVENLABS_API_KEY, @elevenlabs/elevenlabs-js, @elevenlabs/client, text-to-speech, TTS, voice synthesis

**When to use:**

• Generating speech audio from text (narration, audiobooks, announcements)
• Streaming audio in real-time for low-latency playback
• Cloning voices from audio samples (instant or professional voice cloning)
• Converting speech from one voice to another (speech-to-speech)
• Building real-time conversational AI agents with voice interaction
• Controlling pronunciation with SSML or pronunciation dictionaries
• Generating audio with character-level timestamp alignment

**Key patterns covered:**

• Client initialization and configuration (retries, timeouts, API key)
• Text-to-speech conversion and streaming (`convert`, `stream`, timestamps)
• Voice settings (`stability`, `similarityBoost`, `style`, `speed`)
• Voice selection and management (`voices.search`, `voices.get`)
• Voice cloning (instant via `voices.ivc.create`)
• Speech-to-speech voice conversion
• WebSocket input streaming for real-time text-to-speech
• Pronunciation dictionaries and SSML
• Conversational AI agents (`@elevenlabs/client`)
• Model selection, output formats, error handling

**When NOT to use:**

• You need multi-provider voice AI (multiple TTS vendors) -- use a unified abstraction
• You only need browser-side audio playback without generation -- use the Web Audio API
• You need speech-to-text transcription only -- ElevenLabs has this, but it is a separate concern

---

Examples Index

• [Core: Setup, TTS, Streaming & Voice Settings](examples/core.md) -- Client init, convert, stream, timestamps, voice settings, output formats
• [Voices & Cloning](examples/voices.md) -- Voice search, selection, instant voice cloning, speech-to-speech
• [WebSocket & Conversational AI](examples/websocket.md) -- WebSocket input streaming, conversational AI agents, real-time patterns
• [Quick API Reference](reference.md) -- Model IDs, method signatures, output formats, error types, voice settings

---

<philosophy>

Philosophy

The ElevenLabs SDK provides **direct access to the most advanced voice AI API** available. It wraps the ElevenLabs REST API with full TypeScript types, streaming support, and automatic retries.

**Core principles:**

1. **Streams everywhere** -- All audio methods return `ReadableStream<Uint8Array>`. You pipe them to files, HTTP responses, or audio players. The SDK never buffers entire audio files in memory. 2. **Voice settings are the primary control surface** -- `stability`, `similarityBoost`, `style`, and `speed` shape every generation. Learn these four knobs well. 3. **Model selection drives the quality/latency tradeoff** -- `eleven_v3` for best quality, `eleven_flash_v2_5` for sub-75ms latency, `eleven_multilingual_v2` for stable long-form. 4. **Two packages for two use cases** -- `@elevenlabs/elevenlabs-js` for server-side TTS/voice management, `@elevenlabs/client` for browser-side conversational AI agents. 5. **Built-in resilience** -- The SDK retries on 408, 409, 429, and 5xx errors (2 retries by default) with configurable timeouts.

</philosophy>

---

<patterns>

Core Patterns

Pattern 1: Client Setup

Initialize the ElevenLabs client. It auto-reads `ELEVENLABS_API_KEY` from the environment.

// lib/elevenlabs.ts -- basic setup
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

const client = new ElevenLabsClient();
export { client };

// lib/elevenlabs.ts -- production configuration
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

const TIMEOUT_SECONDS = 60;
const MAX_RETRIES = 3;

const client = new ElevenLabsClient({
  apiKey: process.env.ELEVENLABS_API_KEY,
  timeoutInSeconds: TIMEOUT_SECONDS,
  maxRetries: MAX_RETRIES,
});

export { client };

**Why good:** Minimal setup, env var auto-detected, named constants for production settings

// BAD: Hardcoded API key
const client = new ElevenLabsClient({
  apiKey: "sk-1234567890abcdef",
});

**Why bad:** Hardcoded API key is a security breach risk, will leak in version control

**See:** [examples/cor