client-tools

Client Tools

Extend your agent with custom capabilities. Tools let the agent take actions beyond just talking.

Tool Types

| Type | Execution | Use Case | |------|-----------|----------| | **Webhook** | Server-side via HTTP | Database queries, API calls, secure operations | | **Client** | Browser-side JavaScript | UI updates, local storage, navigation | | **System** | Built-in ElevenLabs | End call, transfer, standard actions |

Where Tools Live

Tools are defined inside `conversation_config.agent.prompt`. Webhook and client tools go in the `tools` array. System tools go in `built_in_tools`:

conversation_config={
    "agent": {
        "prompt": {
            "prompt": "You are helpful.",
            "llm": "gemini-2.0-flash",
            "tools": [...],            # Webhook and client tools
            "built_in_tools": {...}     # System tools (end_call, transfer, etc.)
        }
    }
}

Webhook Tools

Execute server-side logic when the agent needs external data or actions.

Basic Webhook

agent = client.conversational_ai.agents.create(
    name="Weather Assistant",
    conversation_config={
        "agent": {
            "prompt": {
                "prompt": "You are a helpful assistant that can check the weather.",
                "llm": "gemini-2.0-flash",
                "tools": [{
                    "type": "webhook",
                    "name": "get_weather",
                    "description": "Get current weather for a city. Use when user asks about weather.",
                    "api_schema": {
                        "url": "https://api.example.com/weather",
                        "method": "POST",
                        "request_headers": {
                            "Authorization": "Bearer {{API_KEY}}"
                        },
                        "request_body_schema": {
                            "type": "object",
                            "properties": {
                                "city": {
                                    "type": "string",
                                    "description": "City name, e.g., 'San Francisco'"
                                },
                                "units": {
                                    "type": "string",
                                    "enum": ["celsius", "fahrenheit"],
                                    "description": "Temperature units"
                                }
                            },
                            "required": ["city"]
                        }
                    }
                }]
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
    }
)

Webhook Request Format

When the agent calls a webhook tool, ElevenLabs sends:

{
  "tool_call_id": "call_abc123",
  "tool_name": "get_weather",
  "parameters": {
    "city": "San Francisco",
    "units": "fahrenheit"
  },
  "conversation_id": "conv_xyz789"
}

Webhook Response Format

Your server should respond with:

{
  "result": "The weather in San Francisco is 68°F and sunny."
}

Or for structured data:

{
  "result": {
    "temperature": 68,
    "condition": "sunny",
    "humidity": 45
  }
}

Webhook with Authentication

# Inside conversation_config.agent.prompt.tools:
{
    "type": "webhook",
    "name": "lookup_order",
    "description": "Look up order status by order ID",
    "response_timeout_secs": 10,
    "api_schema": {
        "url": "https://api.mystore.com/orders/lookup",
        "method": "POST",
        "request_headers": {
            "Authorization": "Bearer {{ORDER_API_KEY}}",
            "X-Store-ID": "store_123"
        },
        "request_body_schema": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "Order ID (e.g., ORD-12345)"
                }
            },
            "required": ["order_id"]
        }
    }
}

Use workspace environment variables to keep a single server tool configuration working across staging and production. `{{system_env__label}}` works in server tool URLs, secret environment variables can populate `request_headers`, and auth-connection environment variables can populate `api_schema.auth_connection`. The same environment-variable resolution model also applies to MCP server connections.

{
  "api_schema": {
    "url": "https://{{system_env__api_host}}.example.com/orders",
    "method": "GET",
    "request_headers": {
      "X-Api-Key": { "env_var_label": "orders_api_key" }
    },
    "auth_connection": { "env_var_label": "orders_oauth" }
  }
}

Workspace auth connections support OAuth2 client credentials, OAuth2 JWT, private key JWT, basic auth, bearer auth, and custom header auth.

System dynamic variables are also available in tool parameters and headers. Use `{{system__conversation_history}}` when a webhook or sub-agent needs the full conversation context as a lazily evaluated JSON history object with user, agent, and tool entries.

Webhook Tool Options

| Field | Type | Default | Description | |-------|------|---------|-------------| | `response_timeout_secs` | int | `20` | Timeout in seconds (5-120) | | `disable_interruptions` | bool | `false` | Prevent user interruptions during tool execution | | `execution_mode` | string | `"immediate"` | `immediate`, `post_tool_speech`, or `async` | | `tool_call_sound` | string | - | Sound during execution: `typing`, `elevator1`-`elevator4` | | `force_pre_tool_speech` | bool | `false` | Force agent to speak before executing tool | | `tool_error_handling_mode` | string | `"auto"` | `auto`, `summarized`, `passthrough`, or `hide` |

**Note:** The default `api_schema.method` is `GET`. Always set `"method": "POST"` explicitly for webhook tools that send request bodies.

Server Implementation (Node.js)

app.post("/webhook/get_weather", async (req, res) => {
  const