/ai

name: ai
description: Use when calling the app's AI gateway from agent tools — chat completions, embeddings, listing models, configuring defaults or BYOK, reading token/cost usage, polling video jobs or preparing declared fictional image references

Butterbase AI Gateway

The AI gateway provides chat, embeddings, model listing, configuration, usage reporting, and asynchronous video jobs. One umbrella tool: **`manage_ai`**.

| Action | What it does | Returns | |---|---|---| | `chat` | Synchronous chat completion (no streaming) | OpenAI-shaped `{ choices: [...] }` | | `embed` | Vector embeddings for string or string[] | OpenAI-shaped `{ data: [{ embedding: [...] }] }` | | `list_models` | Available models with capabilities | `{ models: AiModel[] }` | | `get_config` | Current AI config (default model, BYOK key flag, etc.) | `AiConfig` | | `update_config` | Set defaults, allowed models, max tokens, BYOK | `AiConfig` | | `get_usage` | Token + cost aggregate over a window | usage record | | `submit_video` | Submit an ordinary async video job | `{ job_id, status, polling_url }` | | `poll_video` | Read an existing video job without submitting again | status, optional stage and authenticated content URLs |

---

1. Chat

manage_ai({
  action: "chat",
  app_id,
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user",   content: "What's RAG?" }
  ],
  model: "openai/gpt-4o-mini",     // optional — falls back to app's default
  temperature: 0.2,                // optional
  max_tokens: 500                  // optional
})

This action sets `stream: false` deliberately — agent tools don't stream. If you need partial-token deltas, drive the SDK's `ai.chatStream(…)` from inside a function or DO instead.

`messages[].content` can be a string or an array of content parts (`{ type: "text", text }`, `{ type: "image_url", image_url: {...} }`, `{ type: "video_url", video_url: {...} }`).

---

2. Embed

manage_ai({
  action: "embed",
  app_id,
  input: "hello world",            // or ["a", "b", "c"]
  model: "openai/text-embedding-3-small",   // optional
  encoding_format: "float"          // or "base64"
})

---

3. List models

manage_ai({ action: "list_models", app_id })
// → { models: [{ id, provider, capabilities: ["chat", "embed", ...], context_window, pricing }, ...] }

Use this to discover what the app can call — capabilities + context window matter when picking a model.

---

4. Configure

manage_ai({
  action: "update_config",
  app_id,
  config: {
    defaultModel: "openai/gpt-4o-mini",
    allowedModels: ["openai/gpt-4o-mini", "anthropic/claude-haiku-4-5"],
    maxTokensPerRequest: 4000,
    byokKey: "..." // optional — rotates the customer-supplied OpenRouter / Anthropic key
  }
})

• `maxTokensPerRequest` is server-clamped to 1–100000.
• `allowedModels` is a whitelist — empty means all models the provider exposes.
• Setting `byokKey` switches the app to route through that customer key. Clear it by passing `byokKey: ""` (returns to platform pool).

---

5. Usage

manage_ai({
  action: "get_usage",
  app_id,
  startDate: "2026-05-01",
  endDate:   "2026-05-31"
})

Returns aggregate token counts + cost. Useful for billing reconciliation, spending-cap diagnostics, and showing dashboards.

---

6. Common pitfalls

• **Trying to stream from a tool** — `manage_ai` is synchronous. Use the SDK inside a function for streamed deltas.
• **Sending `stream: true` in the body** — the tool ignores it; always wired to `false`.
• **Hardcoding `model`** — better to omit, let the app's `defaultModel` win, and surface that knob via `update_config`.
• **Skipping `list_models` before suggesting one** — model availability shifts; verify before recommending.

---

7. What this skill does NOT cover

• Streaming chat — use the SDK (`ai.chatStream`) inside a function or DO.
• Vector storage / retrieval — see `butterbase-skills:rag-dev` (RAG collections wrap embeddings + search together).
• AI in deployed functions — they import `@butterbase/sdk` and call `client.ai.*` directly; no MCP needed at runtime.

---

8. Decisions (typed classification)

For routing, classification, moderation or scoring, where code needs a choice / yes-no / score with probabilities, use `manage_ai` `action: "decide"` (SDK `ai.decide`) with a decision model (default `typesafe/jev-1.13`) instead of asking a chat model for JSON. It is cheaper, faster and returns typed probabilities. Full reference: `butterbase_docs` topic `ai`, "Decision models".

---

9. Video jobs and declared fictional references

Before video work, call `butterbase_docs` with `topic: "ai"` and discover supported models with `manage_ai({ action: "list_models", app_id, modality: "video" })`.

Preserve the returned job ID, show the reported stage, poll without issuing another submission, and display authenticated content URLs after completion. Use the caller's explicit fictional-character declaration only. Never silently add declarations or automatically label real people as fictional.

Inspect the connected `manage_ai` tool schema before choosing a client surface. When it includes both `reference_declarations` and `prepare_references`, add the declaration example below to `manage_ai` with `action: "submit_video"`. For older MCP servers that lack these fields, send the fields in the REST body for `POST /v1/:appId/videos/completions`; do not assume an older wrapper forwards unknown arguments. Poll the returned job ID with `poll_video` or `GET /v1/:appId/videos/completions/:jobId`. Do not invent an SDK video helper.

Optional fictional character references

Eligible apps and models can declare specific image slots as AI-generated fictional characters:

{
  "input_images": ["https://images.example/fictional-character.png"],
  "reference_declarations": [
    {"path": "/input_images/0", "kind": "ai_generated_fictional_character"}
  ],
  "prepare_references": "on