Zuplo AI Gateway
Zuplo's AI Gateway acts as an intelligent proxy layer that sits between your engineering team's apps and LLM providers like OpenAI, Anthropic, Google, Mistral, and xAI. Instead of your apps communicating directly with these providers, all requests flow through the Zuplo AI Gateway, which streams responses while applying policies, controls, and monitoring.
Each piece of software that calls the gateway registers as an app—your support chatbot is one app; an internal coding agent is another. A single AI Gateway holds as many apps as you need, and you control exactly which policies run for each one. Every app has its own ordered policy chain, so you can give the customer-facing chatbot an app-specific budget and guardrails while leaving those controls out of the internal agent's chain. Beyond the built-in policies, you can write your own in TypeScript, and apps add them to their chains like any other policy. Chain changes apply within about a minute, with no redeploy. See Apps.
Key Benefits
Provider Independence: Switch between LLM providers (OpenAI, Anthropic, Google, Mistral, xAI, and more) dynamically without modifying app code. Configure model access through the gateway rather than hard coding it into your apps.
Cost Control: Set spending limits at gateway, team, and app levels with hierarchical budgets that cascade down through your structure. Configure daily and monthly thresholds with enforcement or warning notifications.
Security & Compliance: Apply guardrails to detect and block prompt injection attempts and prevent PII leakage in both requests and responses through integrated AI firewall policies.
Self-Service Access: Developers can create apps and access LLMs without needing direct access to provider API keys. Administrators configure providers once, and teams consume them securely.
Extensibility: Write custom policies in TypeScript—content filters, dynamic model routing, plan-based limits—and let apps add them to their chains like any built-in policy.
Performance Optimization: Enable semantic caching to identify and return cached responses for similar prompts, reducing costs and improving response times.
Full Observability: Real-time dashboards show request counts, token usage, time-to-first-byte metrics, and spending patterns across your gateway.
How It Works
Your code sends its LLM requests to your AI Gateway instead of directly to a provider. Each app has its own URL, API key, and policy chain, so one gateway serves teams with completely different configurations.
Each request arrives at the URL of the app it belongs to. The gateway identifies that app, runs its policy chain (model access, app-specific cost controls, security guardrails), routes to the selected LLM provider, and streams the response back. Throughout this process, the gateway captures metrics without exposing underlying provider credentials.
Every app-specific control is opt-in. An app whose policy chain is empty runs no app-selected controls: each request names its own model, and no app-specific model restrictions, budgets, guardrails, or caching apply. Gateway and team usage limits still apply.
Core Features
Multi-Provider Support
Configure multiple LLM providers within a single AI Gateway project. Supported
providers include OpenAI, Anthropic, Google, Mistral, xAI, and OpenAI-compatible
custom providers. See AI Providers for the full list of
providers and supported capabilities. Apps reference models as
providerName/model—for example openai/gpt-5-mini—so a single app can use
models from several providers.
Source-Controlled Gateway
Every AI Gateway project connects to a Git repository that contains the gateway's routes and policy declarations. The default branch is what production runs, so gateway changes go through your normal review workflow. See Source Control.
Per-App Policy Chains
Each app runs its own ordered chain of policies—model filtering, fallback models, app-specific budgets, semantic caching, guardrails, tracing, and any custom policies declared in the repository. Team policy templates give new apps a consistent starting pipeline. See Policy Chains and Custom Policies.
Team Hierarchy & Budgets
Organize users into teams with hierarchical structures. Set budget limits at each level that cascade down:
- Gateway: Limits across the Zuplo project (for example, $1,000/day)
- Teams: Team-specific limits that can't exceed parent limits (for example, $500/day for the Credit Team)
- Apps: Per-app limits for granular control
See Usage Limits.
App Configuration
Each app gets its own:
- Unique Gateway URL: Single endpoint regardless of underlying provider
- API Key: Zuplo-managed key that never exposes provider credentials
- Policy Chain: Model access, app-specific budgets, caching, guardrails, and custom policies, applied in the order the app chooses
Zuplo accounts and projects
A Zuplo account is the top-level container for your members and projects. Each Zuplo project belongs to one account, and an AI Gateway is a type of Zuplo project. The providers, teams, and apps you configure for a gateway all belong to that project.
Account roles can grant access across the projects in a Zuplo account, while project roles grant access to one Zuplo project. AI Gateway team roles add access to a team and its apps within that project. See Role Permissions for the exact permissions each role grants.
Use Cases
- Multi-tenant AI Apps: Enforce spending limits per customer or team
- Agent Development: Build AI agents that can switch providers without code changes
- Cost Management: Control and monitor LLM spending across your gateway
- Security Compliance: Ensure PII and prompt injection protection across all LLM interactions
- Performance: Reduce costs and latency with semantic caching for common queries