/model-routing

Model Routing System

How Routing Decisions Are Made

Every user prompt goes through a 9-tier classification pipeline before any AI model processes it. The system answers three questions:

1. **Which model should handle this?** — 9-tier cost/complexity classification 2. **Is the classifier itself working?** — Cascading fallback (qwen3 → kimi → deepseek → cache) 3. **Can we verify the result?** — Tool-level fallback + auto-evaluation

The Pipeline

User types prompt
    ↓
UserPromptSubmit hook fires (~/.claude/hooks/route-task-hook)
    ↓
Classifier: qwen3 (local, free) classifies into tier
    ↓  (fails?)
Classifier: kimi (local, free) retries
    ↓  (fails?)
Classifier: deepseek-flash (~$0.0001) retries
    ↓  (fails?)
Classifier: cached tier from last success
    ↓
Hook injects routing decision into Claude's context
    ↓
Claude delegates to the right model or handles directly

9-Tier Routing Table

| Tier | Model | Input (per M) | Output (per M) | Handles | |------|-------|---------------|----------------|---------| | 0 | **Qwen3** (local) | $0 | $0 | grep, find, shell, syntax, log reading | | 1 | **Gemini 2.5 Flash-Lite** | $0.10 | $0.40 | Bulk extraction, classification, CIG pipelines | | 2 | **DeepSeek V4 Flash** | $0.14 | $0.28 | Simple code, CRUD, test writing, small fixes | | 3 | **DeepSeek V4 Pro** | $0.44 | $0.87 | Multi-file features, refactors, debugging (~80% of work) | | 4 | **Gemini 2.5 Flash** | $0.15 | $0.60 | Multimodal (images, video, audio), brand analysis | | 5 | **Kimi K2.6** | $0.60 | $2.50 | Code review, commit messages, diff summaries | | 6 | **Gemini 3.1 Pro + Search** | $1.25 | $10.00 | Deep research, Google grounding, 2M context | | 7 | **Codex** | varies | varies | Bulk generation, code review | | 8 | **Claude Sonnet/Opus** | $3-5 | $15-25 | Architecture, security, quality-critical |

Delegation Commands

When the hook says "delegate to X", run the matching command and return its output:

# Tier 0 — Qwen3
~/bin/qwen3 "prompt"

# Tier 1 — Gemini Flash-Lite
~/bin/gemini --flash-lite "prompt"

# Tier 2 — DeepSeek Flash
~/bin/deepseek --flash "prompt"

# Tier 3 — DeepSeek Pro
~/bin/deepseek --pro "prompt"

# Tier 4 — Gemini Flash
~/bin/gemini --flash "prompt"

# Tier 5 — Kimi
~/bin/kimi --quiet -p "prompt"

# Tier 6 — Gemini Pro Search
~/bin/gemini --pro-search "prompt"

# Tier 7 — Codex
codex exec "prompt"

# Tier 8 — Claude
# Handle directly (no delegation)

Delegation Script Contract

Every `~/bin/` script follows the same pattern:

1. **Accepts prompt as argument**: `script "what is 2+2"` 2. **Model flags**: `--flash`, `--pro`, `--flash-lite`, `--pro-search` 3. **Quiet mode**: `--quiet` (where applicable) 4. **Output**: writes response to stdout, errors to stderr 5. **Exit codes**: 0 on success, non-zero on failure

Available Scripts

~/bin/
├── qwen3       # Shell: curl to local Ollama API
├── kimi        # Shell: execs Kimi CLI binary
├── deepseek    # Python: httpx to DeepSeek Anthropic-compat API
├── gemini      # Python: httpx to Gemini OpenAI-compat API
├── research    # Python: multi-backend research with auto-evaluation
└── route-task  # Shell: qwen3-powered task classification

Classifier Fallback Chain

The classifier itself can fail. When it does, cascading fallback kicks in:

| Level | Classifier | Cost | Threshold | |-------|-----------|------|-----------| | 1 | **qwen3** (Ollama) | $0 | 2s connect, 8s classify | | 2 | **kimi** CLI | $0 | Local process | | 3 | **deepseek-flash** | ~$0.0001 | API call | | 4 | **Cached tier** | $0 | From `~/.claude/routing-cache.json` |

The cache (`~/.claude/routing-cache.json`) saves the last successful tier and timestamp. After compaction, when Ollama may be briefly unreachable, the cache ensures routing continues without dropping to CLAUDE by default.

Tool Fallback Protocol

When Claude's built-in tools fail, external backends take over:

| Failed Tool | Fallback 1 | Fallback 2 | |-------------|------------|------------| | **WebSearch** / **WebFetch** | `~/bin/research "query"` | `~/bin/deepseek --pro "query"` | | **Read** / file access | `cat` via Bash | — | | **Grep** | `grep -r` via Bash | — |

Research Tool (`~/bin/research`)

Multi-backend research with auto-evaluation:

• Tries **deepseek-flash → deepseek-pro** in sequence
• Scores results 0-10 on content quality, structure, length
• Auto-adjusts preferred backend based on evaluation scores
• View stats: `~/bin/research --eval`
• Score log: `~/.claude/research-eval.jsonl`

Maggy Integration

Maggy's `model_router.py` mirrors the same 9-tier structure in `DEFAULT_TIERS`. The `PiAdapter` uses the same delegation scripts for execution. Task type overrides in `routing_rules_defaults.py` ensure:

• `research`, `competitor` → **Gemini Pro Search** (Google grounding)
• `bulk` → **Gemini Flash-Lite** (cheapest)
• `security`, `architecture`, `planning` → **Claude** (quality-critical)
• `docs`, `tests` → **DeepSeek Pro** (cost-efficient)
• `review` → **Claude** (security + architecture depth)

Environment

# Required for delegation scripts (in ~/.zshrc)
export DEEPSEEK_API_KEY="sk-..."
export GEMINI_API_KEY="..."       # For gemini delegator
export OPENAI_API_KEY="sk-..."    # For codex CLI

# Ollama must be running locally for qwen3
ollama serve  # or launch at startup

Observability

• **Routing log**: `~/.claude/routing-log.jsonl` — every classification with tier, classifier used, tokens saved
• **Routing cache**: `~/.claude/routing-cache.json` — last tier for post-compact recovery
• **Research eval**: `~/.claude/research-eval.jsonl` — per-query backend scoring
• **Maggy routing heatmap**: Dashboard → Models tab → per-model reward scores