/ai-infrastructure-huggingface-inference

name: ai-infrastructure-huggingface-inference
description: Hugging Face Inference SDK patterns for TypeScript/Node.js — InferenceClient setup, chat completion, text generation, streaming, embeddings, image generation, audio transcription, translation, summarization, and Inference Endpoints

Hugging Face Inference Patterns

> **Quick Guide:** Use `@huggingface/inference` (v4+) to access 200k+ ML models on the Hugging Face Hub. Use `InferenceClient` with `chatCompletion()` for OpenAI-compatible chat, `textGeneration()` for raw text completion, `chatCompletionStream()` for streaming, `featureExtraction()` for embeddings, `textToImage()` for image generation, and `automaticSpeechRecognition()` for audio transcription. Set `provider` to route through inference providers (Cerebras, Together, Groq, etc.) or use `endpointUrl` for dedicated Inference Endpoints.

---

<critical_requirements>

CRITICAL: Before Using This Skill

**(You MUST always pass an access token to `InferenceClient` -- never deploy without authentication)**

**(You MUST use `chatCompletion()` / `chatCompletionStream()` for conversational LLM tasks -- these follow the OpenAI-compatible message format)**

**(You MUST handle errors using `InferenceClientError` and its subclasses -- never use bare catch blocks without error type checking)**

**(You MUST specify a `model` parameter for every inference call -- there is no default model)**

**(You MUST never hardcode access tokens -- always use environment variables via `process.env.HF_TOKEN`)**

</critical_requirements>

---

**Auto-detection:** Hugging Face, huggingface, @huggingface/inference, InferenceClient, HfInference, hf.chatCompletion, hf.textGeneration, hf.featureExtraction, hf.textToImage, hf.automaticSpeechRecognition, hf.translation, hf.summarization, hf.textToSpeech, chatCompletionStream, textGenerationStream, HF_TOKEN, inference provider, Inference Endpoints

**When to use:**

• Accessing any of the 200k+ models hosted on the Hugging Face Hub
• Running chat completion with open-source LLMs (Qwen, Mistral, Llama, etc.)
• Generating embeddings with sentence-transformer models for semantic search
• Generating images from text prompts (FLUX, Stable Diffusion)
• Transcribing audio with automatic speech recognition models
• Running translation, summarization, text classification, or NER tasks
• Deploying models on dedicated Inference Endpoints for production use
• Using third-party inference providers (Cerebras, Together, Groq, Replicate, etc.) through a unified API

**Key patterns covered:**

• InferenceClient initialization and configuration
• Chat Completion API (OpenAI-compatible messages format, streaming)
• Text generation (raw completion, streaming)
• Embeddings via feature extraction
• Image generation (text-to-image)
• Audio transcription (automatic speech recognition)
• Translation, summarization, and text classification
• Inference Endpoints (dedicated deployments)
• Inference Providers (routing through third-party services)
• Error handling with typed error classes

**When NOT to use:**

• If you only use OpenAI models -- use the OpenAI SDK directly
• If you need a provider-agnostic unified SDK with structured outputs and tool calling -- use a higher-level AI SDK
• If you need to fine-tune or train models -- use the `@huggingface/hub` package or Python `transformers`

---

Examples Index

• [Core: Setup, Chat & Text Generation](examples/core.md) -- Client init, chat completion, text generation, streaming, error handling
• [Tasks: Embeddings, Vision, Audio & NLP](examples/tasks.md) -- Feature extraction, image generation, speech recognition, translation, summarization, classification
• [Quick API Reference](reference.md) -- Method signatures, error types, provider list, model recommendations

---

<philosophy>

Philosophy

The `@huggingface/inference` SDK provides a **unified TypeScript client** for accessing hundreds of thousands of ML models through multiple backends: serverless Inference Providers, dedicated Inference Endpoints, and local servers.

**Core principles:**

1. **Model-agnostic access** -- One client, any model on the Hub. Swap models by changing the `model` parameter without code changes. 2. **Provider flexibility** -- Route inference through 20+ providers (Cerebras, Together, Groq, Replicate, etc.) with a single `provider` parameter, or deploy your own Inference Endpoints. 3. **Task-oriented API** -- Methods map to ML tasks (`chatCompletion`, `textToImage`, `automaticSpeechRecognition`), not raw HTTP endpoints. 4. **OpenAI-compatible chat** -- `chatCompletion()` uses the OpenAI message format (`role` + `content`), making migration between providers easy. 5. **Streaming as async generators** -- `chatCompletionStream()` and `textGenerationStream()` return `AsyncGenerator`, consumed with `for await...of`.

</philosophy>

---

<patterns>

Core Patterns

Pattern 1: Client Setup

Initialize with your Hugging Face access token. The token is required for authenticated access.

// lib/hf-client.ts -- basic setup
import { InferenceClient } from "@huggingface/inference";

const client = new InferenceClient(process.env.HF_TOKEN);

export { client };

// lib/hf-client.ts -- with custom endpoint
const ENDPOINT_URL =
  "https://your-endpoint.us-east-1.aws.endpoints.huggingface.cloud/v1/";

const client = new InferenceClient(process.env.HF_TOKEN, {
  endpointUrl: ENDPOINT_URL,
});

export { client };

**Why good:** Token from env var, named constant for endpoint URL, named export

// BAD: Hardcoded token, no named export
const hf = new InferenceClient("hf_abc123xyz");
export default hf;

**Why bad:** Hardcoded token is a security risk, default export violates conventions

**See:** [examples/core.md](examples/core.md) for provider routing, local endpoints, and endpoint helper

---

Pattern 2: Chat Completion (OpenAI-Compatible)

Use `chatCompletion()` for conversational LLM tasks.