agent-configuration

Agent Configuration

Complete reference for configuring conversational AI agents.

Configuration Structure

agent = client.conversational_ai.agents.create(
    name="My Agent",
    conversation_config={
        "agent": {
            "first_message": "Hello!",
            "language": "en",
            "prompt": {           # LLM, system prompt, tools, and knowledge base
                "prompt": "You are helpful.",
                "llm": "gemini-2.0-flash",
                "tools": [...],
                "built_in_tools": {...}
            }
        },
        "tts": {...},             # Voice and TTS model settings
        "asr": {...},             # Speech recognition settings
        "turn": {...},            # Turn-taking behavior
        "conversation": {...},    # Duration, events, monitoring
        "vad": {...},             # Voice activity detection config
        "language_presets": {...}  # Language-specific overrides
    },
    platform_settings={...}       # Auth, call limits
)

conversation_config

Controls the real-time conversation behavior.

agent

conversation_config={
    "agent": {
        "first_message": "Hello! How can I help you today?",
        "language": "en",
        "disable_first_message_interruptions": False,
        "prompt": {
            "prompt": "You are a helpful assistant.",
            "llm": "gemini-2.0-flash",
            "temperature": 0.7
        }
    }
}

| Field | Type | Default | Description | |-------|------|---------|-------------| | `first_message` | string | `""` | What the agent says when conversation starts | | `language` | string | `"en"` | ISO 639-1 language code (en, es, fr, etc.) | | `disable_first_message_interruptions` | bool | `false` | Prevent user from interrupting the first message | | `hinglish_mode` | bool | `false` | When enabled and language is Hindi, agent responds in Hinglish | | `dynamic_variables` | object | - | Config with `dynamic_variable_placeholders` containing key-value pairs | | `prompt` | object | - | LLM configuration (see prompt section below) |

tts (Text-to-Speech)

conversation_config={
    "tts": {
        "voice_id": "JBFqnCBsd6RMkjVDRZzb",
        "model_id": "eleven_flash_v2_5",
        "stability": 0.5,
        "similarity_boost": 0.8,
        "speed": 1.0,
        "optimize_streaming_latency": 3,
        "expressive_mode": True
    }
}

| Field | Type | Default | Description | |-------|------|---------|-------------| | `voice_id` | string | `"cjVigY5qzO86Huf0OWal"` | Voice to use | | `model_id` | string | - | TTS model (see below) | | `stability` | float | `0.5` | 0-1, lower = more expressive | | `similarity_boost` | float | `0.8` | 0-1, higher = closer to original voice | | `speed` | float | `1.0` | 0.7-1.2, speech speed multiplier | | `optimize_streaming_latency` | int | - | 0-4, higher = faster but lower quality | | `expressive_mode` | bool | `true` | Enable expressive voice generation | | `agent_output_audio_format` | string | - | Output audio codec format | | `pronunciation_dictionary_locators` | array | - | Pronunciation overrides |

**Available TTS models for agents:**

| Model ID | Languages | Latency | |----------|-----------|---------| | `eleven_flash_v2_5` | 32 | ~75ms (recommended) | | `eleven_flash_v2` | English | ~75ms | | `eleven_turbo_v2_5` | 32 | ~250-300ms | | `eleven_turbo_v2` | English | ~250-300ms | | `eleven_multilingual_v2` | 29 | Standard | | `eleven_v3_conversational` | 70+ | Standard |

asr (Automatic Speech Recognition)

conversation_config={
    "asr": {
        "quality": "high",
        "keywords": ["ElevenLabs", "TechCorp"],
        "user_input_audio_format": "pcm_16000"
    }
}

| Field | Type | Default | Description | |-------|------|---------|-------------| | `quality` | string | `"high"` | Transcription quality level | | `provider` | string | `"elevenlabs"` | ASR provider (`elevenlabs` or `scribe_realtime`) | | `keywords` | array | - | Words to boost recognition accuracy | | `user_input_audio_format` | string | - | Input audio format (e.g., `pcm_16000`, `ulaw_8000`) |

turn (Turn-Taking)

conversation_config={
    "turn": {
        "turn_timeout": 7,
        "turn_eagerness": "normal",
        "silence_end_call_timeout": -1
    }
}

| Field | Type | Default | Description | |-------|------|---------|-------------| | `turn_timeout` | number | `7` | Seconds to wait before re-engaging the user | | `turn_eagerness` | string | `"normal"` | How quickly agent responds: `patient`, `normal`, or `eager` | | `silence_end_call_timeout` | number | `-1` | Seconds of silence before ending call (-1 = disabled) | | `initial_wait_time` | number | - | Seconds to wait for user to start speaking | | `spelling_patience` | string | `"auto"` | Entity detection patience: `auto` or `off` | | `speculative_turn` | bool | `false` | Enable speculative turn detection | | `soft_timeout_config` | object | - | Configures a message if user is silent (see below) |

**soft_timeout_config:**

| Field | Type | Default | Description | |-------|------|---------|-------------| | `timeout_seconds` | number | `-1` | Seconds before soft timeout (-1 = disabled) | | `message` | string | `"Hhmmmm...yeah."` | What agent says on timeout | | `use_llm_generated_message` | bool | `false` | Let LLM generate the timeout message |

prompt (nested in conversation_config.agent)

Configures the LLM behavior. This object lives at `conversation_config.agent.prompt`:

conversation_config={
    "agent": {
        "prompt": {
            "prompt": "You are a helpful customer service agent...",
            "llm": "gemini-2.0-flash",
            "temperature": 0.7,
            "max_tokens": 500,
            "tools": [...],
            "built_in_tools": {...},
            "knowledge_base": [...]
        }
    }
}

| Field | Type | Default | Description | |-------|------|---------|-------------| | `prompt` | string | `""` |