/ai-infrastructure-together-ai

name: ai-infrastructure-together-ai
description: Together AI SDK patterns for TypeScript — client setup, chat completions, streaming, structured output, function calling, embeddings, image generation, fine-tuning, and OpenAI-compatible endpoints

Together AI SDK Patterns

> **Quick Guide:** Use the `together-ai` npm package to access 200+ open-source models (Llama, Qwen, Mistral, DeepSeek) via Together AI's fast inference API. The SDK mirrors the OpenAI API shape -- `client.chat.completions.create()` for chat, `client.images.generate()` for images, `client.embeddings.create()` for embeddings. Use `response_format: { type: "json_schema" }` with Zod-generated schemas for structured output. Function calling uses the same `tools` parameter shape as OpenAI. You can also use the OpenAI SDK directly by pointing `baseURL` to `https://api.together.xyz/v1`.

---

<critical_requirements>

CRITICAL: Before Using This Skill

**(You MUST use the `together-ai` package (`import Together from "together-ai"`) -- NOT the OpenAI SDK -- unless explicitly building an OpenAI-compatible integration)**

**(You MUST include the JSON schema in BOTH the `response_format` parameter AND the system prompt when using structured output -- the model needs both)**

**(You MUST handle errors using `Together.APIError` and its subclasses -- never use bare catch blocks without error type checking)**

**(You MUST never hardcode API keys -- always use environment variables via `process.env.TOGETHER_API_KEY`)**

</critical_requirements>

---

**Auto-detection:** Together AI, together-ai, together.ai, TOGETHER_API_KEY, client.chat.completions (together), client.images.generate, client.embeddings.create (together), Llama-3, Qwen3, Mistral, DeepSeek, FLUX, together.images, together.chat, together.embeddings, together.fineTuning, api.together.xyz

**When to use:**

• Running open-source LLMs (Llama, Qwen, Mistral, DeepSeek) via serverless inference
• Generating images with FLUX or Stable Diffusion models
• Creating embeddings for RAG pipelines with open-source embedding models
• Using function calling / tool use with open-source models
• Extracting structured JSON output from LLM responses
• Fine-tuning open-source models on custom data
• Migrating from OpenAI to open-source models with minimal code changes

**Key patterns covered:**

• Client initialization and configuration (retries, timeouts, logging)
• Chat completions with open-source models (Llama, Qwen, Mistral, DeepSeek)
• Streaming with `stream: true` and `for await...of`
• Structured output with `response_format: { type: "json_schema" }` and Zod
• Function calling / tool use with `tools` parameter
• Image generation with FLUX and Stable Diffusion models
• Embeddings API with open-source embedding models
• Fine-tuning API (file upload, job creation, monitoring)
• OpenAI SDK compatibility (base URL swap)
• Error handling, retries, timeouts

**When NOT to use:**

• You need OpenAI-specific features (Responses API, Batch API, Realtime API) -- use the OpenAI SDK directly
• You want framework-specific chat UI hooks -- use a framework-integrated AI SDK
• You only use OpenAI models and never plan to use open-source models

---

Examples Index

• [Core: Setup & Configuration](examples/core.md) -- Client init, production config, error handling, OpenAI compatibility
• [Chat Completions](examples/chat.md) -- Basic chat, multi-turn, model selection, vision
• [Streaming](examples/streaming.md) -- Async iteration, stream cancellation
• [Tool/Function Calling](examples/tools.md) -- Tool definitions, multi-step tool loops
• [Structured Output](examples/structured-output.md) -- JSON mode, Zod schemas, regex mode
• [Images & Embeddings](examples/images.md) -- FLUX image generation, embedding models, semantic search
• [Quick API Reference](reference.md) -- Model IDs, method signatures, error types

---

<philosophy>

Philosophy

Together AI provides **fast serverless inference for open-source models**. The TypeScript SDK (`together-ai`) is auto-generated with Stainless and mirrors the OpenAI API shape, making migration straightforward.

**Core principles:**

1. **OpenAI-compatible API shape** -- Same `client.chat.completions.create()` pattern, same `messages` array, same `tools` parameter. Switching from OpenAI is often just changing the import and model name. 2. **Open-source model access** -- Run Llama, Qwen, Mistral, DeepSeek, and 200+ other models without managing infrastructure. Models are identified by their Hugging Face-style IDs (e.g., `meta-llama/Llama-3.3-70B-Instruct-Turbo`). 3. **Multi-modal support** -- Chat completions, image generation (FLUX, Stable Diffusion), embeddings, audio, and video -- all through one SDK. 4. **Structured output via JSON Schema** -- Pass a JSON schema in `response_format` and include it in the system prompt. Use Zod's `z.toJSONSchema()` to generate schemas from TypeScript types. 5. **Fine-tuning open-source models** -- Upload JSONL data, create LoRA or full fine-tuning jobs, and deploy custom models -- all via the API.

**When to use Together AI:**

• You want to use open-source models with fast serverless inference
• You need cost-effective inference (often cheaper than proprietary APIs)
• You want to fine-tune open-source models on your data
• You need image generation with FLUX models
• You want OpenAI API compatibility for easy migration

**When NOT to use:**

• You need OpenAI-specific features (Responses API, Batch API, Realtime) -- use the OpenAI SDK
• You need Anthropic or Google-specific features -- use their respective SDKs
• You want a provider-agnostic SDK -- use a unified provider framework

</philosophy>

---

<patterns>

Core Patterns

Pattern 1: Client Setup

Initialize the Together client. It reads `TOGETHER_API_KEY` from the environment.

// lib/together.ts -- basic setup
import Together from "together-ai";
const client = new Together();
export { client };

// lib/together.ts --