An open-source API gateway that sits between your applications and multiple LLM providers, routing requests to providers such as OpenAI, Anthropic and Google Vertex AI through one middleware layer.
export LLM_GATEWAY_SECRET="$(openssl rand -base64 32 | tr -d '\n')" export GATEWAY_API_KEY_HASH_SECRET="$(openssl rand -base64 32 | tr -d '\n')" ./scripts/run-unified-container.shdocker volume create llmgateway_postgres docker volume create llmgateway_redis docker run -d \ --name llmgateway \ --restart unless-stopped \ -p 3002:3002 \ -p 3003:3003 \ -p 3005:3005 \ -p 3006:3006 \ -p 3007:3007 \ -p 4001:4001 \ -p 4002:4002 \ -v llmgateway_postgres:/var/lib/postgresql/data \ -v llmgateway_redis:/var/lib/redis \ -e AUTH_SECRET="$LLM_GATEWAY_SECRET" \ -e GATEWAY_API_KEY_HASH_SECRET="$GATEWAY_API_KEY_HASH_SECRET" \ ghcr.io/theopenco/llmgateway-unified:latestpnpm i && pnpm run setup$ pnpm dev$ pnpm build
You can use LLM Gateway in two ways:
Hosted Version: For immediate use without setup, visit llmgateway.io to create an account and get an API key.
Self-Hosted: Deploy LLM Gateway on your own infrastructure for complete control over your data and configuration.
Self-Hosted With Docker
Use Docker-managed volumes for the unified image.
export LLM_GATEWAY_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
export GATEWAY_API_KEY_HASH_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
./scripts/run-unified-container.shIf you prefer a one-off docker run, use named volumes instead of /some-host-dir:
docker volume create llmgateway_postgres
docker volume create llmgateway_redis
docker run -d \
--name llmgateway \
--restart unless-stopped \
-p 3002:3002 \
-p 3003:3003 \… 10 more lines
Using LLM Gateway API
curl -X POST https://api.llmgateway.io/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "Hello, how are you?"}
]… 1 more line
LLM_GATEWAY_SECRETGATEWAY_API_KEY_HASH_SECRETAUTH_SECRETLLM_GATEWAY_API_KEYcurl -X POST https://api.llmgateway.io/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "Hello, how are you?"}
]
}'Read before runningLLM Gateway is an open-source API gateway for Large Language Models (LLMs). It acts as a middleware between your applications and various LLM providers, allowing you to:
You can use LLM Gateway in two ways:
Use Docker-managed volumes for the unified image. Do not bind-mount a host directory directly to /var/lib/postgresql/data, because PostgreSQL initialization inside the container needs to set permissions on that directory and that can fail depending on the host filesystem and ownership.
export LLM_GATEWAY_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
export GATEWAY_API_KEY_HASH_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
./scripts/run-unified-container.sh
If you prefer a one-off docker run, use named volumes instead of ~/some-host-dir:
docker volume create llmgateway_postgres
docker volume create llmgateway_redis
docker run -d \
--name llmgateway \
--restart unless-stopped \
-p 3002:3002 \
-p 3003:3003 \
-p 3005:3005 \
-p 3006:3006 \
-p 3007:3007 \
-p 4001:4001 \
-p 4002:4002 \
-v llmgateway_postgres:/var/lib/postgresql/data \
-v llmgateway_redis:/var/lib/redis \
-e AUTH_SECRET="$LLM_GATEWAY_SECRET" \
-e GATEWAY_API_KEY_HASH_SECRET="$GATEWAY_API_KEY_HASH_SECRET" \
ghcr.io/theopenco/llmgateway-unified:latest
curl -X POST https://api.llmgateway.io/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
-d '{
"model": "gpt-4o",
"messages": [
{"role": "user", "content": "Hello, how are you?"}
]
}'
Install dependencies and set up the development environment:
pnpm i && pnpm run setup
This will install all dependencies, start Docker services, sync the database schema, and seed initial data.
Note for WSL2 users: Ensure Docker Desktop is running with WSL integration enabled.
Start development servers:
pnpm dev
Build for production:
pnpm build
apps/ui: Next.js dashboard frontendapps/playground: Lounge — consumer AI chat app (Next.js)apps/code: Next.js Dev Plans + coding tools landing & dashboardapps/airside: Airside — self-serve provider portal (Next.js)apps/api: Hono backendapps/gateway: API gateway for routing LLM requestsapps/docs: Documentation siteee/admin: Internal admin dashboardpackages/db: Drizzle ORM schema and migrationspackages/models: Model and provider definitionspackages/shared: Shared types and utilitiesWe do not accept pull requests that add new providers. Inference providers onboard through Airside, our self-serve provider portal, where you register and manage your model listings.
LLMGateway is available under a dual license:
ee/ directory require an Enterprise license; multi-organization administration requires a white-label license. See ee/LICENSE for details.For enterprise licensing, please contact us at contact@llmgateway.io
Stars on GitHub
A development methodology for coding agents that makes the agent brainstorm before building, follow plans, run parallel work and finish branches in a disciplined way. You get fewer skipped steps and more consistent results from your coding assistant.
@obraHelps you take a change from a rough idea or bug report through to a tested, reviewed result, with each step handled by a focused skill. Its skills cover pinning down domain terms, designing module interfaces, diagnosing hard bugs, and reviewing changes against your repo's coding standards.
@mattpocockGives your coding agent a planning, testing and review routine to follow on every task. It plans before building, checks changes with tests, has a fresh reviewer look for regressions, and saves repeated wins as reusable skills.
@affaan-mDeveloper tools for working on the React codebase itself. Provides skills for extracting error messages, managing and debugging feature flags across release channels, running Flow type checks, and fixing lint and formatting issues before committing.
@facebookAn open-source API gateway that sits between your applications and multiple LLM providers, routing requests to providers such as OpenAI, Anthropic and Google Vertex AI through one middleware layer. Install it with the command on its page. Free and open source.
Is this tool yours?
Claim it with GitHubSubmit a pluginPromote it