# AI Gateway overview

Zuplo's AI Gateway acts as an intelligent proxy layer that sits between your
engineering team's apps and LLM providers like OpenAI, Anthropic, Google,
Mistral, and xAI. Instead of your apps communicating directly with these
providers, all requests flow through the Zuplo AI Gateway, which streams
responses while applying policies, controls, and monitoring.

Each piece of software that calls the gateway registers as an **app**—your
support chatbot is one app; an internal coding agent is another. A single AI
Gateway holds as many apps as you need, and you control exactly which policies
run for each one. Every app has its own ordered policy chain, so you can give
the customer-facing chatbot an app-specific budget and guardrails while leaving
those controls out of the internal agent's chain. Beyond the built-in policies,
you can write your own in TypeScript, and apps add them to their chains like any
other policy. Chain changes apply within about a minute, with no redeploy. See
[Apps](./apps.mdx).

## Key Benefits

**Provider Independence**: Switch between LLM providers (OpenAI, Anthropic,
Google, Mistral, xAI, and more) dynamically without modifying app code.
Configure model access through the gateway rather than hard coding it into your
apps.

**Cost Control**: Set spending limits at gateway, team, and app levels with
hierarchical budgets that cascade down through your structure. Configure daily
and monthly thresholds with enforcement or warning notifications.

**Security & Compliance**: Apply guardrails to detect and block prompt injection
attempts and prevent PII leakage in both requests and responses through
integrated AI firewall policies.

**Self-Service Access**: Developers can create apps and access LLMs without
needing direct access to provider API keys. Administrators configure providers
once, and teams consume them securely.

**Extensibility**: Write custom policies in TypeScript—content filters, dynamic
model routing, plan-based limits—and let apps add them to their chains like any
built-in policy.

**Performance Optimization**: Enable semantic caching to identify and return
cached responses for similar prompts, reducing costs and improving response
times.

**Full Observability**: Real-time dashboards show request counts, token usage,
time-to-first-byte metrics, and spending patterns across your gateway.

## How It Works

Your code sends its LLM requests to your AI Gateway instead of directly to a
provider. Each app has its own URL, API key, and policy chain, so one gateway
serves teams with completely different configurations.

Each request arrives at the URL of the app it belongs to. The gateway identifies
that app, runs its policy chain (model access, app-specific cost controls,
security guardrails), routes to the selected LLM provider, and streams the
response back. Throughout this process, the gateway captures metrics without
exposing underlying provider credentials.

Every app-specific control is opt-in. An app whose policy chain is empty runs no
app-selected controls: each request names its own model, and no app-specific
model restrictions, budgets, guardrails, or caching apply. Gateway and team
usage limits still apply.

## Core Features

### Multi-Provider Support

Configure multiple LLM providers within a single AI Gateway project. Supported
providers include OpenAI, Anthropic, Google, Mistral, xAI, Amazon Bedrock
(through [Bedrock Mantle](./bedrock-mantle.mdx)), Google Cloud
([Vertex AI](./vertex-ai.mdx)), and OpenAI-compatible custom providers. See
[AI Providers](./providers.mdx) for the full list of providers and supported
capabilities. Apps reference models as `providerName/model`—for example
`openai/gpt-5-mini`—so a single app can use models from several providers.

### Source-Controlled Gateway

Every AI Gateway project connects to a Git repository that contains the
gateway's routes and policy declarations. The default branch is what production
runs, so gateway changes go through your normal review workflow. See
[Source Control](./source-control.mdx).

### Per-App Policy Chains

Each app runs its own ordered chain of policies—model filtering, fallback
models, app-specific budgets, semantic caching, guardrails, tracing, and any
custom policies declared in the repository. Team policy templates give new apps
a consistent starting pipeline. See [Policy Chains](./policy-chains.mdx) and
[Custom Policies](./custom-policies.mdx).

### Team Hierarchy & Budgets

Organize users into teams with hierarchical structures. Set budget limits at
each level that cascade down:

- **Gateway**: Limits across the Zuplo project (for example, $1,000/day)
- **Teams**: Team-specific limits that can't exceed parent limits (for example,
  $500/day for the Credit Team)
- **Apps**: Per-app limits for granular control

See [Usage Limits](./usage-limits.mdx).

### App Configuration

Each app gets its own:

- **Unique Gateway URL**: Single endpoint regardless of underlying provider
- **API Key**: Zuplo-managed key that never exposes provider credentials
- **Policy Chain**: Model access, app-specific budgets, caching, guardrails, and
  custom policies, applied in the order the app chooses

### Zuplo accounts and projects

A **Zuplo account** is the top-level container for your members and projects.
Each **Zuplo project** belongs to one account, and an AI Gateway is a type of
Zuplo project. The providers, teams, and apps you configure for a gateway all
belong to that project.

Account roles can grant access across the projects in a Zuplo account, while
project roles grant access to one Zuplo project. AI Gateway team roles add
access to a team and its apps within that project. See
[Role Permissions](../articles/accounts/roles-and-permissions.mdx) for the exact
permissions each role grants.

## Use Cases

- **Multi-tenant AI Apps**: Enforce spending limits per customer or team
- **Agent Development**: Build AI agents that can switch providers without code
  changes
- **Cost Management**: Control and monitor LLM spending across your gateway
- **Security Compliance**: Ensure PII and prompt injection protection across all
  LLM interactions
- **Performance**: Reduce costs and latency with semantic caching for common
  queries