lynkr

Lynkr

An LLM Gateway which optimises your token usage.

84% fewer tokens on JSON tool results. 53% fewer tokens on tool-heavy requests. Sub-300ms semantic cache hits. Zero code changes.

Numbers from the bundled benchmark against LiteLLM on identical free local backends — run it yourself: node benchmark-tier-routing.js. It doubles as a 19-scenario routing regression harness (currently 12/12 correctness checks), and MODE=routing runs a routing-only head-to-head that judges both proxies on the same acceptable-tier sets — including LiteLLM's Auto Router v2. How it works →

Third-party benchmark: on RouterArena (ICLR 2026, 8,400 queries) Lynkr's routing scores 67.65 arena / 68.41% accuracy at $0.29 per 1K queries with 92.38 robustness — above GPT-5's built-in router and NotDiamond at a fraction of their cost. Methodology & caveats →

---

🚀 New: Wrap Mode for AI Coding Tools

Use Lynkr's routing with your AI coding assistant — maximize your subscription value:

npm install -g lynkr


# Claude Code Pro/Max
lynkr wrap claude

Wrapping gives you:

• ✅ Tier routing (send simple tasks to free Ollama, complex to your subscription/API)
• ✅ Sticky sessions: one routing decision per conversation via content fingerprinting, with automatic escalation when the task outgrows the model
• ✅ TOON/RTK compression (84% token reduction on large JSON tool outputs)
• ✅ Semantic caching (sub-300ms cache hits, 0 tokens billed)
• ✅ Your subscription is used only where it counts — simple requests route to your own local/API models, so subscription capacity goes to the work that needs it
• ✅ Works with OAuth (Claude, Copilot, Cursor) or API keys (Aider, Codex)

Full wrap guide →

---

Quick Start (2 Minutes)

1. Install Lynkr

npm install -g lynkr

2. Configure Lynkr

The fastest path is the interactive wizard:

lynkr init

It asks four questions — usage mode (Claude Pro/Max via wrap, or direct API keys), tier picks for SIMPLE/MEDIUM/COMPLEX/REASONING across the 14 supported providers, credentials for what you chose, and a few routing-intelligence knobs — then writes a fully-populated .env with sensible production defaults for everything else (caching, compression, policy budgets, MCP sandbox, rate limiting).

Useful flags:

lynkr init --force                # overwrite an existing .env

See docs/init.md for the full wizard reference.

If you'd rather configure by hand, the manual options below still work — copy .env.example to .env and edit it directly:

Option A: Free & Local (Ollama) - Recommended for Testing

# Install Ollama first: https://ollama.com
ollama pull qwen2.5-coder:latest

Then start Lynkr:

lynkr start

3. Connect Your Tool

Cursor IDE

• Settings → Models → Override Base URL
• Set to: http://localhost:8081/v1
• API Key: any-value

Codex CLI

Edit ~/.codex/config.toml:

model_provider = "lynkr"

[model_providers.lynkr]
base_url = "http://localhost:8081/v1"
wire_api = "responses"

✅ Done! Your AI tool now uses your chosen provider.

---

Common Startup Errors

Error: unable to determine transport target for "pino-pretty"

Problem: You're running an older version (< 9.3.0).

Solution: Update to the latest version:

npm install -g lynkr@latest

If you must use an older version, set NODE_ENV=production before starting.

Warning: Missing tier configuration: TIER_SIMPLE, TIER_MEDIUM...

This is just a warning - you can ignore it. Tier routing is optional.

To remove the warning, add to .env:

TIER_SIMPLE=ollama:qwen2.5-coder:latest
TIER_MEDIUM=ollama:qwen2.5-coder:latest
TIER_COMPLEX=ollama:qwen2.5-coder:latest
TIER_REASONING=ollama:qwen2.5-coder:latest

Warning: FALLBACK_PROVIDER='databricks' is enabled but missing credentials

Solution: Add to .env:

FALLBACK_ENABLED=false

Error: connect ECONNREFUSED ::1:11434 (Ollama)

Problem: Ollama is not running.

Solution:

ollama serve

Keep this terminal open, and start Lynkr in a new terminal.

Error: Connection refused or 404 Not Found

Problem: Lynkr is not running or wrong port.

Solution: Check Lynkr is running on the correct port:

curl http://localhost:8081/

Should return: {"service":"Lynkr","version":"9.x.x","status":"running"}

---

Why Lynkr?

AI coding tools lock you into one provider and send every token raw. Lynkr breaks both locks.

Claude Code / Cursor / Codex / Cline / Continue
                    ↓
                  Lynkr
          ┌─────────────────────┐
          │  Strip unused tools  │  ← 53% fewer tokens on tool calls
          │  Compress JSON blobs │  ← 84% on large tool results
          │  Semantic cache      │  ← <300ms hits, 0 tokens billed
          │  Route by complexity │  ← cheap model for simple, cloud for hard
          │  Learn from outcomes │  ← kNN + bandit + auto-calibration
          └─────────────────────┘
                    ↓
    Ollama | Bedrock | Azure | Moonshot | OpenRouter | OpenAI

What you get:

• ✅ 53% fewer tokens on tool-heavy requests (Claude Code, Cursor sessions)
• ✅ 84% compression on large JSON tool results (grep, file reads, test output)
• ✅ Semantic cache serves repeated queries in under 300ms with 0 tokens billed
• ✅ Automatic tier routing — simple questions go to cheap models, complex ones escalate; sessions stick to one model until the task genuinely outgrows it
• ✅ A closed learning loop — every outcome trains a kNN router and bandit, and tier thresholds re-calibrate nightly from your own traffic
• ✅ Route through your company's infrastructure (Databricks, Azure, Bedrock)
• ✅ Zero code changes — just change one environment variable

---

Supported Providers

Provider	Type	Example Models	Cost
Ollama	Local	qwen2.5-coder, deepseek-coder, llama3	Free
llama.cpp	Local	Any GGUF model	Free
LM Studio	Local	Local models with GUI	Free
OpenRouter	Cloud	GPT-4o, Claude 3.5, Llama 3, Gemini	$
OrcaRouter	Cloud	Chat, reasoning, image, video (one endpoint, live catalog)	$ (zero-markup)
AWS Bedrock	Cloud	Claude, Llama, Mistral, Titan	$$
Databricks	Cloud	Claude Sonnet 4.5, Opus 4.6	$$$
Azure OpenAI	Cloud	GPT-4o, o1, o3	$$$
Azure Anthropic	Cloud	Claude Sonnet, Opus	$$$
OpenAI	Cloud	GPT-4o, o3-mini	$$$
Atlas Cloud	Cloud	Qwen, DeepSeek, and other OpenAI-compatible models	$-$$$
DeepSeek	Cloud	DeepSeek R1, Reasoner	$
Z.ai	Cloud	GLM-4.7, GLM-4.5-Air	$
Moonshot AI	Cloud	Kimi K2.6, Kimi K3	$
Baidu Qianfan	Cloud	ERNIE 4.5 Turbo, ERNIE X1.1	$ (unverified — not yet probed against a live key)
Fireworks AI	Cloud	Kimi K2, GLM-5, DeepSeek V3 (serverless)	$ (unverified — not yet probed against a live key)

4 local providers for 100% offline, free usage. 14+ cloud providers for scale.

---

Advanced: Tier Routing (Save Even More)

Route different request types to different models automatically:

# .env file
MODEL_PROVIDER=ollama
FALLBACK_ENABLED=false

# Use small/fast models for simple tasks
TIER_SIMPLE=ollama:qwen2.5:3b

# Use medium models for normal coding
TIER_MEDIUM=ollama:qwen2.5:7b

# Use powerful models for complex architecture
TIER_COMPLEX=ollama:deepseek-r1:14b
TIER_REASONING=ollama:deepseek-r1:14b

# Optional: Limits (remove for unlimited) for long conversations
POLICY_MAX_STEPS=50
POLICY_MAX_TOOL_CALLS=100

Lynkr analyzes each request and routes it to the appropriate tier. Simple questions use fast models. Complex refactoring uses powerful models. The scorer combines anchor-embedding classification (WS7, payload-invariant) with an LLM difficulty classifier (Phase 6, added 2026-07-19) that catches topic-vs-difficulty confounding — list the exports from this file correctly routes to MEDIUM instead of getting boosted to REASONING by "technical vocabulary" alone.

Result: 70-90% of requests use cheaper/faster models. Only hard problems hit expensive models. 15× reduction in expensive-tier over-routing vs the anchor-only baseline on the 381-prompt eval set (0.6% vs ~15%).

Tier configuration is strictly authoritative — bandit exploration is constrained to the models you've listed in TIER_*, and multi-turn conversations score with a recency-weighted sliding window so context isn't lost on short follow-ups. Conversations get a content-fingerprint session id (clients like Claude Code send none), the decision pins for the session, and a guarded escape ladder (risk keywords, force phrases, score drift, context overflow) re-escalates the moment a task outgrows its model. Full pipeline: docs/routing-intelligence.md · intent scorer: docs/intent-window-routing.md · verify any change: docs/benchmarking.md.

---

Common Issues & Fixes

Issue	Solution
"Service temporarily overloaded"	Ollama model too large for RAM. Use smaller model or increase --max-old-space-size
"Route not found: HEAD /"	Ignore - harmless health check from Claude Code
"Hallucinated tool calls"	Normal - Lynkr automatically filters invalid tools
"Safe Command DSL blocked"	Add POLICY_SAFE_COMMANDS_ENABLED=false to .env
"spawn graphify ENOENT"	Optional feature. Set CODE_GRAPH_ENABLED=false in .env (see Advanced Features section for installation)
Slow first request (20+ sec)	Ollama loading model into memory. Add OLLAMA_KEEP_ALIVE=30m in Ollama config
No response after N turns	Remove POLICY_MAX_STEPS and POLICY_MAX_TOOL_CALLS from .env (unlimited by default in v9.3.0+)

---

Advanced Features

Real-time SSE streaming (all tiers)

Responses stream token-by-token instead of arriving all at once — including through the tier router. Two mechanisms, both on by default:

• Native passthrough: when the upstream already speaks Anthropic SSE