---
title: "AI Gateway — Control LLM Traffic"
description:
  "Every LLM call, on the same gateway as your APIs. Route between providers,
  cache semantically, block prompt injections, and cap dollar spend per team —
  with one programmable policy engine."
canonicalUrl: "https://zuplo.com/ai-gateway"
sourceUrl: "https://zuplo.com/ai-gateway"
pageType: "product"
generatedAt: "2026-10-08"
---

# Every LLM call, on the same gateway as your APIs.

> Right now, nothing is capping the bill or checking the prompts. Route your LLM
> calls through the gateway that already runs your APIs — cap spend per team,
> block abuse, and switch providers without touching code.

## Built for LLM traffic, governed like an API.

- **Multi-provider routing** — Route to OpenAI, Anthropic, Gemini, and Mistral
  through one endpoint. Swap providers in config, not code.
- **Semantic caching** — Cache responses by vector similarity, not exact match.
  Cut latency and spend on repeated prompts.
- **Prompt injection protection** — Malicious instructions get blocked before
  they ever reach your model.
- **Secret masking** — Keys, emails, and PII are redacted before anyone — or
  anything — sees them.
- **Dollar budgets, per team** — Per-team spend caps. Requests halt at the limit
  — no surprise bill.
- **Observability & tracing** — Stream every request to Galileo, Comet Opik, or
  your own collector. Inspect latency, tokens, and cost per call.

## Policies and budgets at every level of your org.

Most AI gateways give you one global cap. Zuplo nests orgs, teams, and apps,
each with its own budget — predictable spend, without anyone asking for a new
key.

- Dollar budgets at org, team, and app level
- Daily and monthly limits — hard stops or warnings
- Sub-team budgets cannot exceed the parent's ceiling
- Rate limits and security policies inherit the same way

## Route to the right model, every time.

One endpoint in front of every provider. Teams keep their SDK; platform teams
decide which model actually serves each call.

- Route to OpenAI, Anthropic, Gemini, or Mistral
- Pin the model per route, tenant, or key
- Keep the OpenAI SDK — just change the base URL
- Works with Claude Code, Cursor, LangChain, Goose

## Swap the base URL. Keep your SDK.

Point any OpenAI-compatible client at your Zuplo gateway and policies take over.
No new SDK to learn.

```typescript
// Keep your existing OpenAI SDK — just swap the base URL
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://your-org.zuplo.app/v1",
  apiKey: process.env.ZUPLO_AI_KEY, // your Zuplo key, not OpenAI's
});

const response = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Summarize our Q1 sales data." }],
});
```

## Not a standalone AI box — part of the unified gateway for APIs, AI, and MCP.

The AI Gateway runs on the same policy engine, auth, and GitOps pipeline as your
REST and MCP traffic. One bill, one audit trail — when your AI product hits
production, the gateway already knows how to run it.

- [Explore the platform](/)
- [See every policy](https://zuplo.com/docs/policies)

## The questions every CISO asks about AI.

Security, cost, and data-privacy controls a centralized AI governance program
needs — and where Zuplo enforces each one.

### Unrestricted access

**How do we prevent models from acting beyond their intended scope?** Every key
is scoped to approved models and tools. The model can't see anything else.
[MCP Gateway](/mcp-gateway)

### Cost control

**How do we cap costs when AI usage exceeds projections?** Dollar budgets per
org, team, and app that hard-stop before the bill arrives. You always know which
team is spending what.
[Usage limits](https://zuplo.com/docs/ai-gateway/usage-limits)

### Data privacy

**How do we keep PII and sensitive data out of logs and prompts?** Sensitive
data is redacted before it reaches the model, and scrubbed again before logs
leave. Two layers, on by default.
[AI Firewall](https://zuplo.com/docs/ai-gateway/policies/akamai-ai-firewall)

### Visibility & monitoring

**How do we monitor AI activity, especially for agents and AIOps?** Every call
is logged with who, what, and how much — streamed to your SIEM. For agents, you
see what they did, not just what they said.
[Audit logs](https://zuplo.com/docs/programmable-api/audit-log)

### Prompt injection

**How do we mitigate "ignore all previous instructions" attacks?** You mitigate,
not eliminate: detect attacks at the door, and cap the damage with tool
allowlists if one gets through.
[AI Firewall](https://zuplo.com/docs/ai-gateway/policies/akamai-ai-firewall)

### Cross-team data leakage

**How do we prevent one team's model from accessing another team's logs?** Each
team gets its own tools and its own logs. Finance sees Stripe, Engineering sees
GitHub — never each other. [MCP Gateway](/mcp-gateway)

### Unauthorized access

**How do we stop developers from bypassing the gateway and calling Bedrock or
OpenAI directly?** Honest answer: no gateway alone does. Zuplo is the
enforcement point and key vault; lock down network egress and IAM, and our
Akamai partnership covers the rest.
[Dedicated deployment](https://zuplo.com/docs/dedicated/akamai/ai-powered-applications)

## FAQ

### How does Zuplo handle rate limiting for LLM API calls?

Zuplo supports request-based rate limiting with the rate-limit-inbound policy
and usage quotas with the quota-inbound policy. Quotas can be keyed by API key,
user, or any custom property and applied per minute, day, or month. Token-level
accounting is implemented via custom TypeScript policies that read the token
counts returned by your LLM provider.

### Can Zuplo act as an AI gateway in front of OpenAI, Anthropic, or other LLM providers?

Yes. Zuplo proxies requests to OpenAI, Anthropic, Gemini, and Mistral with
authentication, rate limiting, cost controls, audit logging, prompt injection
protection, and semantic caching as a transparent layer. Provider routing is
configured declaratively — you can map requests across providers and define
fallback behavior in your policy pipeline.

### How does Zuplo help control AI API costs?

Zuplo reduces AI costs through several built-in mechanisms: semantic caching
returns cached responses for semantically similar queries, the quota-inbound
policy caps consumption per consumer, and hierarchical team budgets set dollar
spend limits. For cost-aware model routing (sending simpler prompts to cheaper
models), teams use custom TypeScript policies — see our 'Control AI Costs'
guide.

### How does Zuplo support token-based billing for AI products?

Zuplo can meter token consumption from LLM API responses and report usage to
billing providers like Stripe. Custom TypeScript policies extract token counts
from provider responses (e.g., OpenAI's usage object) and record them as
billable units. This enables precise usage-based billing that reflects actual
LLM cost drivers.

### How does semantic caching work in Zuplo for AI APIs?

Zuplo's semantic cache uses vector embeddings to identify requests that are
semantically similar — not just identical — to previously cached queries. When a
match is found above a configurable similarity threshold, the cached response is
returned immediately without forwarding to the LLM. This reduces latency, cuts
API costs, and improves throughput for AI-powered applications.

## Next steps

Start with the AI Gateway today. Free forever for developers. Same signup as the
API Gateway — your account owns both.

- [Start for Free](https://portal.zuplo.com/signup)
- [Book a Technical Call](/schedule-call)
- [Compare AI Gateways](/learning-center/best-ai-gateway-buyers-guide.md)