AI Gateway Concepts
The Zuplo AI Gateway is a proxy that sits between your apps and LLM providers like OpenAI, Anthropic, Google, Mistral, and xAI. Instead of each app holding provider API keys and calling providers directly, every request flows through the gateway, which applies policies, controls, and monitoring. This page explains the concepts behind it. For the full feature tour, see the AI Gateway overview.
Providers, teams, and apps
Three building blocks organize an AI Gateway:
- Providers are the LLM vendors your gateway can route to. Administrators configure providers once, including any custom OpenAI-compatible providers, and consumers never see the underlying credentials.
- Teams group users and carry budget limits that cascade down the hierarchy: gateway-wide limits, then team limits, then per-app limits. See Usage Limits.
- Apps are the pieces of software that call the gateway — a support chatbot is one app, an internal coding agent is another. Each app gets its own gateway URL, its own Zuplo-managed API key, and its own policy chain.
Policy chains
Every app runs its own ordered policy chain — model filtering, fallback models, budgets, semantic caching, guardrails, tracing, and any custom policies written in TypeScript. Team policy templates give new apps a consistent starting pipeline, and chain changes apply within about a minute with no redeploy.
App-level controls are opt-in: an app with an empty policy chain applies no model restrictions, budgets, guardrails, or caching of its own — though gateway and team usage limits still apply.
The Universal API
Every app exposes an OpenAI-compatible
Universal API. Point any SDK that supports a
custom base URL at the app's gateway URL, send the app's API key as a bearer
token, and reference models as providerName/model (for example
openai/gpt-5-mini). One app can use models from several providers through the
same endpoint, so switching providers is a configuration change rather than a
code change.
How it fits into Zuplo
An AI Gateway is a type of Zuplo project. Its routes and policy declarations live in a Git repository, so gateway changes go through the same review workflow as the rest of your Zuplo configuration, and the providers, teams, and apps you configure belong to that project. The platform primitives — deployment model, environments, analytics — are the same ones described in How Zuplo Works.
Next steps
- Get started with the AI Gateway
- Browse the SDK integrations for OpenAI, the Vercel AI SDK, LangChain, Claude Code, Codex, and Goose
- Explore the AI Gateway policies