client-tools

Client Tools

Extend your agent with custom capabilities. Tools let the agent take actions beyond just talking.

Tool Types

| Type | Execution | Use Case | |------|-----------|----------| | **Webhook** | Server-side via HTTP | Database queries, API calls, secure operations | | **Client** | Browser-side JavaScript | UI updates, local storage, navigation | | **System** | Built-in ElevenLabs | End call, transfer, standard actions |

Where Tools Live

Tools are defined inside `conversation_config.agent.prompt`. Webhook and client tools go in the `tools` array. System tools go in `built_in_tools`:

conversation_config={
    "agent": {
        "prompt": {
            "prompt": "You are helpful.",
            "llm": "gemini-2.0-flash",
            "tools": [...],            # Webhook and client tools
            "built_in_tools": {...}     # System tools (end_call, transfer, etc.)
        }
    }
}

Webhook Tools

Execute server-side logic when the agent needs external data or actions.

Basic Webhook

agent = client.conversational_ai.agents.create(
    name="Weather Assistant",
    conversation_config={
        "agent": {
            "prompt": {
                "prompt": "You are a helpful assistant that can check the weather.",
                "llm": "gemini-2.0-flash",
                "tools": [{
                    "type": "webhook",
                    "name": "get_weather",
                    "description": "Get current weather for a city. Use when user asks about weather.",
                    "api_schema": {
                        "url": "https://api.example.com/weather",
                        "method": "POST",
                        "request_headers": {
                            "Authorization": "Bearer {{API_KEY}}"
                        },
                        "request_body_schema": {
                            "type": "object",
                            "properties": {
                                "city": {
                                    "type": "string",
                                    "description": "City name, e.g., 'San Francisco'"
                                },
                                "units": {
                                    "type": "string",
                                    "enum": ["celsius", "fahrenheit"],
                                    "description": "Temperature units"
                                }
                            },
                            "required": ["city"]
                        }
                    }
                }]
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
    }
)

Webhook Request Format

When the agent calls a webhook tool, ElevenLabs sends:

{
  "tool_call_id": "call_abc123",
  "tool_name": "get_weather",
  "parameters": {
    "city": "San Francisco",
    "units": "fahrenheit"
  },
  "conversation_id": "conv_xyz789"
}

Webhook Response Format

Your server should respond with:

{
  "result": "The weather in San Francisco is 68°F and sunny."
}

Or for structured data:

{
  "result": {
    "temperature": 68,
    "condition": "sunny",
    "humidity": 45
  }
}

Webhook with Authentication

# Inside conversation_config.agent.prompt.tools:
{
    "type": "webhook",
    "name": "lookup_order",
    "description": "Look up order status by order ID",
    "response_timeout_secs": 10,
    "api_schema": {
        "url": "https://api.mystore.com/orders/lookup",
        "method": "POST",
        "request_headers": {
            "Authorization": "Bearer {{ORDER_API_KEY}}",
            "X-Store-ID": "store_123"
        },
        "request_body_schema": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "Order ID (e.g., ORD-12345)"
                }
            },
            "required": ["order_id"]
        }
    }
}

Use workspace environment variables to keep a single server tool configuration working across staging and production. `{{system_env__label}}` works in server tool URLs, secret environment variables can populate `request_headers`, and auth-connection environment variables can populate `api_schema.auth_connection`. The same environment-variable resolution model also applies to MCP server connections.

{
  "api_schema": {
    "url": "https://{{system_env__api_host}}.example.com/orders",
    "method": "GET",
    "request_headers": {
      "X-Api-Key": { "env_var_label": "orders_api_key" }
    },
    "auth_connection": { "env_var_label": "orders_oauth" }
  }
}

Workspace auth connections support OAuth2 client credentials, OAuth2 JWT, private key JWT, basic auth, bearer auth, custom header auth, and mutual TLS (`mtls`).

System dynamic variables are also available in tool parameters and headers. Use `{{system__conversation_history}}` when a webhook or sub-agent needs the full conversation context as a lazily evaluated JSON history object with user, agent, and tool entries.

Webhook Tool Options

| Field | Type | Default | Description | |-------|------|---------|-------------| | `response_timeout_secs` | int | `20` | Timeout in seconds (5-120) | | `interruption_mode` | string | `"allow"` | Controls whether the user can interrupt around this tool call: `allow`, `disable_during_tool`, or `disable_during_tool_and_turn` | | `execution_mode` | string | `"immediate"` | `immediate`, `post_tool_speech`, or `async` | | `tool_call_sound` | string | - | Sound during execution: `typing`, `elevator1`-`elevator4` | | `pre_tool_speech` | string | `"auto"` | Controls whether the agent speaks before execution: `auto`, `force`, or `off` | | `tool_error_handling_mode` | string | `"auto"` | `auto`, `summarized`, `passthrough`, or `hide` | | `api_schema.response_filter` | object | - | Filters JSON webhook responses before the LLM sees them. Use `mode: "allow