--- title: "AI Gateway: Spend Ceilings, Guardrails, and Source Control" description: "The latest AI Gateway release adds spend limits for gateways, teams, and apps, sensitive-data scanning, source-controlled projects, and more provider options." canonicalUrl: "https://zuplo.com/changelog/2026/08/19/ai-gateway-spend-ceilings-and-guardrails" pageType: "changelog" date: "2026-08-19" tags: "runtime, portal, policy, security" --- This is the biggest AI Gateway update we have shipped so far. It adds spend attribution and limits for apps, teams, and gateways, alongside source-controlled projects, per-app policy chains, sensitive-data scanning, and new provider options. - **Usage limits at every level**: Set daily or monthly limits on an app, a team, or the entire gateway - **Source-controlled projects**: Review routes, policies, and custom code in your own repository - **Prompt and completion guardrails**: Mask or block sensitive data, including values split across streamed responses - **More provider options**: Add Bedrock Mantle and xAI, with updated Google and Mistral support ## Usage limits for apps, teams, and gateways Limits can count cost, tokens, or requests over a daily or monthly period. An app can have its own limit, its parent team can set another, and the gateway can enforce a final limit across every team underneath it. The gateway checks each level on every request and stops at the first exceeded limit. App limits come from the Budgets and Costs policy. Team and gateway limits are enforced outside the app's policy chain, so they still apply if an app owner reorders or disables local policies. Each app chooses what happens next. The default is a `429` response that names the level, limit type, and period that blocked the request. You can also send over-budget traffic to a cheaper fallback model. The Teams & Apps analytics view shows how those limits apply across the full hierarchy. Switch between cost, tokens, and requests, then compare daily or monthly usage at the gateway, team, or app level. ![AI Gateway team analytics showing monthly spend and budget usage across the gateway, teams, and apps](../public/media/changelog/2026-08/ai-gateway-usage-limits.png) See the [usage limits documentation](https://zuplo.com/docs/ai-gateway/usage-limits) for configuration details. ## Source-controlled projects and per-app policy chains Every AI Gateway project now deploys from a repository you own on GitHub, GitLab, Bitbucket, or Azure DevOps. Routes, policy declarations, and custom code can go through pull requests and rollbacks with the rest of your application. `config/policies.json` declares the policies available in a project. In the Portal, each app selects which of those policies run and in what order. An app owner can reorder or disable an existing policy without a deployment, while a new policy or code change still goes through source control. Teams can also define a starting policy chain for new apps. Entries can be locked when a policy must remain in place, and sub-teams inherit the template from their parent. ![Policies tab for an AI Gateway app showing authentication, quota, and custom policies in execution order](../public/media/changelog/2026-08/ai-gateway-policy-chain.png) Read about the [repository layout](https://zuplo.com/docs/ai-gateway/source-control), [policy chains](https://zuplo.com/docs/ai-gateway/policy-chains), and [policy templates](https://zuplo.com/docs/ai-gateway/policy-templates). ## Custom model routing in TypeScript A custom policy is a TypeScript module in your repository. Once declared in `config/policies.json`, it can be added to an app's chain like a built-in policy. `AIGatewayModels.load()` returns the provider catalog with model capabilities, status, and per-token pricing, without exposing credentials. `AIGatewayModelRouting.set()` selects the provider and model for the current request. Together, they support price-aware routing, per-app model selection, and fallback orders defined in code. The [dynamic model routing cookbook](https://zuplo.com/docs/ai-gateway/cookbooks/dynamic-model-routing) includes a working example. See [custom policies](https://zuplo.com/docs/ai-gateway/custom-policies) for the authoring model. ## Sensitive-data detection for prompts and completions The new Data Loss Prevention policy runs inside the gateway and scans both prompts and completions. It includes more than 80 rule types for PII, financial data, national identifiers, network addresses, and credentials from providers such as OpenAI, Anthropic, AWS, Stripe, and GitHub. Each rule can mask, block, log, or ignore a match. Inbound and outbound actions are configured separately, and the scanner catches values split across chunks in a streamed response. The project template starts with PII and card numbers masked and secrets blocked in both directions. The [Data Loss Prevention policy documentation](https://zuplo.com/docs/policies/ai-gateway-dlp-inbound) lists every rule and action. ## Content guardrails with Akamai AI Firewall The Akamai AI Firewall policy inspects prompts before they reach the provider and completions before they return to the caller. Streaming responses are supported. The policy now fails closed by default. If the guardrail service is unavailable, the request returns a `502` instead of continuing without inspection. Set `onError` when availability should take priority for a particular app. See the [Akamai AI Firewall policy](https://zuplo.com/docs/policies/akamai-ai-firewall-v2-inbound) for setup and options. ## Bedrock Mantle, xAI, and updated provider support Bedrock Mantle and xAI are now available in the provider catalog. Bedrock Mantle supports OpenAI-compatible chat completions and embeddings, plus Claude on the native Messages endpoint. xAI provides Grok chat completions and embeddings under `xai/`. Google and Mistral now use their own OpenAI-compatible endpoints. Google requests can pass through tool calls, multimodal input, and JSON Schema structured output. Mistral requests support tools and JSON response formats. You can also add the same provider more than once. Each configuration gets its own name, so separate OpenAI keys with different rate limits or cost centers can coexist in one gateway. The [provider documentation](https://zuplo.com/docs/ai-gateway/providers) has the full list of supported models and endpoints. ## Conversation-aware semantic caching Semantic caching can now include several recent messages in its cache key. Set `recentMessageCount` to control how much of the latest conversation is considered, and use `maxConversationLength` to skip the cache when earlier context would make a match unsafe. Entries are already separated by model, sampling parameters, and system prompt. Cache hits and misses now appear in the request log. See the [semantic cache policy](https://zuplo.com/docs/policies/ai-gateway-semantic-cache-v2-inbound) for configuration details. ## Observability in one place Logs and analytics now share an Observability tab with cost, token, request, and cache breakdowns. Apps and teams appear in one tree, and each app includes a Playground for test requests plus copyable Chat Completions, Responses, and Messages endpoint URLs. Tracing policies for [Comet Opik](https://zuplo.com/docs/policies/comet-opik-tracing-v2-inbound) and [Galileo](https://zuplo.com/docs/policies/galileo-tracing-v2-inbound) are also available in the project template. ## Getting started Create an AI Gateway in the Portal and send a request with one of the keyless demo models, or scaffold the project locally: ```bash npx create-zuplo-api@latest my-ai-gateway --template ai-gateway ``` Follow the [AI Gateway quickstart](https://zuplo.com/docs/ai-gateway/getting-started) for the first request. The [AI Gateway documentation](https://zuplo.com/docs/ai-gateway/introduction) covers providers, teams, apps, usage limits, and policies. For a closer look at how attribution and hierarchical limits work, read [Attribute AI Spend to Teams and Apps](/blog/attribute-ai-spend-teams-apps).