/model-registry-maintainer

name: model-registry-maintainer
description: Guide for maintaining the MassGen model and backend registry. This skill should be used when adding new models, updating model information (release dates, pricing, context windows), or ensuring the registry stays current with provider releases. Covers both the capabilities registry and the pricing/token manager.

Model Registry Maintainer

This skill provides guidance for maintaining MassGen's model registry across two key files:

1. **`massgen/backend/capabilities.py`** - Models, capabilities, release dates 2. **`massgen/token_manager/token_manager.py`** - Pricing, context windows

When to Use This Skill

• New model released by a provider
• Model pricing changes
• Context window limits updated
• Model capabilities changed
• New provider/backend added

Two Files to Maintain

File 1: capabilities.py (Models & Features)

**What it contains:**

• List of available models per provider
• Model capabilities (web search, code execution, vision, etc.)
• Release dates
• Default models

**Used by:**

• Config builder (`--quickstart`, `--generate-config`)
• Documentation generation
• Backend validation

**Always update this file** for new models.

File 2: token_manager.py (Pricing & Limits)

**What it contains:**

• Hardcoded pricing/context windows for models NOT in LiteLLM database
• On-demand loading from LiteLLM database (500+ models)

**Used by:**

• Cost estimation
• Token counting
• Context management

**Pricing resolution order:** 1. LiteLLM database (fetched on-demand, cached 1 hour) 2. Hardcoded PROVIDER_PRICING (fallback only) 3. Pattern matching heuristics

**Only update PROVIDER_PRICING if:**

• Model is NOT in LiteLLM database
• LiteLLM pricing is incorrect/outdated
• Model is custom/internal to your organization

Information to Gather for New Models

1. Release Date

• Format: `"YYYY-MM"`
• Sources:
• OpenAI: https://openai.com/index
• Anthropic: https://www.anthropic.com/news
• Google DeepMind: https://blog.google/technology/google-deepmind/
• xAI: https://x.ai/news

2. Context Window

• Input context size (tokens)
• Max output tokens
• Look for: "context window", "max tokens", "input/output limits"

3. Pricing

• Input cost per 1K tokens (USD)
• Output cost per 1K tokens (USD)
• Cached input cost (if applicable)
• Sources:
• OpenAI: https://openai.com/api/pricing/
• Anthropic: https://www.anthropic.com/pricing
• Google: https://ai.google.dev/pricing
• xAI: https://x.ai/api/pricing

4. Capabilities

• Web search, code execution, vision, reasoning, etc.
• Check official API documentation

5. Model Name

• Exact API identifier (case-sensitive)
• Check provider's model documentation

Adding a New Model - Complete Workflow

Step 1: Add to capabilities.py

Add model to the `models` list and `model_release_dates`:

# massgen/backend/capabilities.py

"openai": BackendCapabilities(
    # ... existing fields ...
    models=[
        "new-model-name",  # Add here (newest first)
        "gpt-5.1",
        # ... existing models ...
    ],
    model_release_dates={
        "new-model-name": "2025-12",  # Add here
        "gpt-5.1": "2025-11",
        # ... existing dates ...
    },
)

Step 2: Check if pricing is in LiteLLM (Usually Skip)

**First, check if the model is already in LiteLLM database:**

import requests

url = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
pricing_db = requests.get(url).json()

if "new-model-name" in pricing_db:
    print("✅ Model found in LiteLLM - no need to update token_manager.py")
    print(f"Pricing: ${pricing_db['new-model-name']['input_cost_per_token']*1000}/1K input")
else:
    print("❌ Model NOT in LiteLLM - need to add to PROVIDER_PRICING")

**Only if NOT in LiteLLM**, add to `PROVIDER_PRICING`:

# massgen/token_manager/token_manager.py

PROVIDER_PRICING: Dict[str, Dict[str, ModelPricing]] = {
    "OpenAI": {
        # Format: ModelPricing(input_per_1k, output_per_1k, context_window, max_output)
        "new-model-name": ModelPricing(0.00125, 0.01, 300000, 150000),
        # ... existing models ...
    },
}

**Provider name mapping**:

• `"OpenAI"` (not "openai")
• `"Anthropic"` (not "claude")
• `"Google"` (not "gemini")
• `"xAI"` (not "grok")

Step 3: Update Capabilities (if new features)

If the model introduces new capabilities:

supported_capabilities={
    "web_search",
    "code_execution",
    "new_capability",  # Add here
}

Step 4: Update Default Model (if appropriate)

Only change if the new model should be the recommended default:

default_model="new-model-name"

Step 5: Validate and Test

# Run capabilities tests
uv run pytest massgen/tests/test_backend_capabilities.py -v

# Test config generation with new model
massgen --generate-config ./test.yaml --config-backend openai --config-model new-model-name

# Verify the config was created successfully
cat ./test.yaml

Step 6: Regenerate Documentation

uv run python docs/scripts/generate_backend_tables.py
cd docs && make html

Current Model Data

OpenAI Models (as of Nov 2025)

In capabilities.py:

models=[
    "gpt-5.1",        # 2025-11
    "gpt-5-codex",    # 2025-09
    "gpt-5",          # 2025-08
    "gpt-5-mini",     # 2025-08
    "gpt-5-nano",     # 2025-08
    "gpt-4.1",        # 2025-04
    "gpt-4.1-mini",   # 2025-04
    "gpt-4.1-nano",   # 2025-04
    "gpt-4o",         # 2024-05
    "gpt-4o-mini",    # 2024-07
    "o4-mini",        # 2025-04
]

In token_manager.py (add missing models):

"OpenAI": {
    "gpt-5": ModelPricing(0.00125, 0.01, 400000, 128000),
    "gpt-5-mini": ModelPricing(0.00025, 0.002, 400000, 128000),
    "gpt-5-nano": ModelPricing(0.00005, 0.0004, 400000, 128000),
    "gpt-4o": ModelPricing(0.0025, 0.01, 128000, 16384),
    "gpt-4o-mini": ModelPricing(0.00015, 0.0006, 128000, 1638