model-message-stream-protocol

MMSP - Model Message Stream Protocol

Integrate every model the same way, and keep one API in your head instead of one per provider.

MMSP, the Model Message Stream Protocol, takes the per-provider differences off a developer's mind: one message format and one streaming grammar for every model provider, in Python and TypeScript.

📖 Documentation: mmsp.penguin.ooo

Design: the rules MMSP keeps and the decisions that changed it, one RFC each, live in design/.

Using a coding agent? Install the MMSP SKILL files from skills/ so it can use MMSP correctly in generated code.

📢 Follow us on or join our

Why MMSP?

• 🔗 Unified: A consistent and intuitive interface for developing agents across different LLMs.

• 🎯 Precise: Automatically handles interleaved thinking during multi-step tool calls, preventing performance degradation.

• 🧭 Traceable: Provides lightweight yet fine-grained tracing for debugging and auditing LLM executions.

Features

AutoLLMClient (Python & TypeScript)

Switch different LLMs with zero code changes and no performance loss.

Built-in Observability

Audit LLM executions by adding a single trace_id parameter, no database required.

Supported Models

Clients

AutoLLMClient takes a model id and a client_type. An official client speaks its vendor's own API, knows the vendor's models, and reads the vendor's key from the environment; a compatible client speaks one wire protocol for any endpoint that serves it.

client_type may be omitted for a model id that begins with a known family: gpt- and text-embedding- route to openai-official, claude- to anthropic-official, gemini- to google-official, glm- to zai-official, kimi- to moonshot-official, deepseek- to deepseek-official, minimax- to minimax-official. Any other id raises and asks for a client_type. The CLIENT_TYPE environment variable names one for every client the code does not.

Gemini on Google Vertex AI takes client_type="google-genai" and the service-account JSON key as the API key: Vertex AI's Interactions endpoint, which google-official speaks, serves none of these models.

Claude on Amazon Bedrock takes client_type="anthropic-official", base_url="bedrock://<region>" and "ACCESS_KEY,SECRET_KEY" as the API key, with a Bedrock inference profile as the model id, such as global.anthropic.claude-opus-5-5.

TokenDance serves Chat Completions, Responses and Embeddings at https://tokendance.space/gateway/v1 and Anthropic Messages at https://tokendance.space/gateway, all with one key.

Where a gateway serves more than one protocol, prefer "openai-responses": OpenRouter serves it for every model it hosts, while SiliconFlow serves Chat Completions only.

The full machine-readable list — model, base URL, client, input/output modalities, context window, and per-million-token list pricing in USD or CNY:

from mmsp import list_supported_models

models = list_supported_models(currency="CNY")  # "USD" by default

import { listSupportedModels } from "@prismshadow/mmsp";

const models = listSupportedModels("CNY"); // "USD" by default

Installation

Python package

Install from PyPI:

uv add mmsp
# or
pip install mmsp

Build from source:

cd src_py && make

See src_py/README.md for comprehensive usage examples and API documentation.

TypeScript package

Install from npm:

npm install @prismshadow/mmsp

Build from source:

cd src_ts && make install && make build

See src_ts/README.md for comprehensive usage examples and API documentation.

Agent Skills

MMSP provides Codex/Claude Code skill files for assistants that need to help users consume the SDK packages:

• Python skill: skills/mmsp-python/SKILL.md
• TypeScript skill: skills/mmsp-typescript/SKILL.md

APIs

AutoLLMClient is the main class for interacting with the MMSP SDK. It is constructed with model and client_type (see Clients; a model id of a known family names its official client on its own), plus optional api_key, base_url, and default_headers — headers sent with every request, for endpoints that demand their own. It provides the following methods:

A key goes only where it was given for. A key passed in is used, with the base_url passed in, else the client's <P>_BASE_URL (see Clients), else its default endpoint. Without a key, a client reads its <P>_API_KEY only when it is given no base_url or its own default endpoint: deepseek-official with no base_url or https://api.deepseek.com uses DEEPSEEK_API_KEY. Any other base_url without a key raises ApiKeyRequiredError at construction: pass the key with it, or drop it and set <P>_API_KEY, plus <P>_BASE_URL for an endpoint other than the default. Two clients are the exception, because their servers may take no key: mmsp and openai-chat-vllm-adapter call a base_url given without a key keyless, and their default endpoint too when <P>_API_KEY is unset (http://127.0.0.1:25752/v1, and http://127.0.0.1:8000/v1, where vllm serve answers); the vLLM adapter sends vLLM's placeholder key EMPTY. The rest of OpenAI's environment (OPENAI_ORG_ID, OPENAI_PROJECT_ID, OPENAI_ADMIN_KEY, OPENAI_WEBHOOK_SECRET and the headers OPENAI_CUSTOM_HEADERS lists) goes only where OPENAI_API_KEY goes: a client built on the OpenAI SDK sends none of it to a gateway, a vLLM server or another vendor's API. @prismshadow/mmsp/credentials and mmsp.credentials export the rule as data for hosts.

• (async) streaming_response(messages, config): Streams the response of LLMs in a stateless manner.
• (async) streaming_response_stateful(message, config): Streams the response of LLMs in a stateful manner.
• (async) list_models(): Lists the model ids the configured endpoint serves. A protocol client (openai-chat, openai-chat-vllm-adapter, openai-responses, ant-messages, openai-embedding, google-genai, mmsp) is named explicitly and lists everything the endpoint serves; a client deduced from a model id lists only the ids that deduce back to it.
• clear_history(): Clears the history of the stateful LLM client.
• get_history(): Returns the history of the stateful LLM client.
• set_history(history): Replaces the history of the stateful LLM client with a copy of the provided list.