AI gateway

Add Search & AI > LiteLLM. The gateway holds your provider keys, and your apps talk to it with a key of its own. Your real keys never reach your app’s code.

chatbot

running

main@e90b4a1

OPENAI_BASE_URL

summaries

running

main@3f2e1c9

OPENAI_API_KEY

ai-gateway

running

OpenAI · Anthropic

keys stay here

Apps talk to the gateway with its own key; your provider keys stay in the gateway.

Set it up

1. Pick the providers you use: OpenAI, Anthropic, Google Gemini, OpenRouter, Mistral, Groq.
2. Put each provider’s key in the gateway node’s own environment: OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY and so on.
3. Link the gateway to your apps and deploy.

What your app gets

Variable	Value
OPENAI_BASE_URL, OPENAI_API_BASE	http://<gateway>:4000/v1
OPENAI_API_KEY	The gateway’s key (sk-…), not your provider key
LLM_GATEWAY_URL	The gateway’s root address, without /v1

Apps reading ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY get the root address and the gateway key, so the Anthropic SDK works too. LITELLM_URL, LITELLM_API_KEY and LLM_API_KEY are also covered.

Any OpenAI SDK works unchanged; ask for a model as provider/model:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_BASE_URL and OPENAI_API_KEY
reply = client.chat.completions.create(
    model="anthropic/claude-sonnet-5-5",
    messages=[{"role": "user", "content": "Hello"}],
)

Model aliases

Model map gives short names to models (up to 20), for example fast = openai/gpt-4o-mini. Change the alias later and every app follows without a redeploy of its code.

Spend cap

Monthly budget (USD) caps spending over 30 days. LiteLLM only enforces budgets with a database: link a Postgres to the gateway (so its env has DATABASE_URL) before you set one.

All linked apps share one gateway key. If you need separate keys per app, use one gateway per app.

The AI gateway may require a paid plan.