# AI Gateway Configuration Executor Policy :::note{title="AI Gateway Policy"} This policy is for use with the [AI Gateway](/docs/ai-gateway/overview). See the AI Gateway documentation to learn how to configure and govern AI models with Zuplo. ::: The AI Gateway Configuration Executor loads each application's configuration (when auth or the configuration loader has not already) and runs an ordered inbound policy chain assembled from policies declared by the gateway. Applications may also include authentication in their own chain. This makes it possible to offer different routing, caching, guardrail, metering, and tracing behavior from one AI Gateway deployment. Prefer placing `ai-gateway-configuration-loader-inbound` before this executor on the route so loading and chain execution stay separate. When the loader is omitted, this executor still loads configuration itself. The gateway owner decides which policy declarations are available, while application and team templates control which entries an application may edit or remove. Each request carries the resulting chain and policy options. ## Configuration The configuration shows how to configure the policy in the 'policies.json' document. ```json title="config/policies.json" { "name": "my-ai-gateway-configuration-executor-inbound-policy", "policyType": "ai-gateway-configuration-executor-inbound", "handler": { "export": "AIGatewayConfigurationExecutorInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "cacheTtlSeconds": 10 } } } ``` ### Policy Configuration - `name` <string> - The name of your policy instance. This is used as a reference in your routes. - `policyType` <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be `ai-gateway-configuration-executor-inbound`. - `handler.export` <string> - The name of the exported type. Value should be `AIGatewayConfigurationExecutorInboundPolicy`. - `handler.module` <string> - The module containing the policy. Value should be `$import(@zuplo/runtime)`. - `handler.options` <object> - The options for this policy. [See Policy Options](#policy-options) below. ### Policy Options The options for this policy are specified below. All properties are optional unless specifically marked as required. - `cacheTtlSeconds` <number> - The time in seconds to cache app configurations loaded by app_id. Defaults to 10 seconds when omitted. Higher values decrease latency; lower values pick up portal changes sooner. Cached results remain valid until the cache expires even if the configuration changes in the portal. This cache is only used when neither ai-gateway-auth-inbound nor ai-gateway-configuration-loader-inbound already loaded the configuration for the request. Defaults to `10`. ## Using the Policy The AI Gateway Configuration Executor loads each application's configuration (when auth or `ai-gateway-configuration-loader-inbound` has not already) and runs its ordered inbound policy chain. Configuration comes from route-level `ai-gateway-auth-inbound` or the configuration loader when either already ran, otherwise from the route's `app_id` path parameter via Gateway Service. Applications can still require API keys later by including `ai-gateway-auth-inbound` in their own `inboundPolicyChain`. Prefer placing the dedicated configuration loader before this executor on the route. When the loader is omitted, this executor still loads configuration itself before running the chain. The gateway owner remains in control: - Every selectable policy must be declared in `config/policies.json`. - Application and team policy templates determine which entries applications may edit or remove before the resulting chain reaches the gateway. ## How application chains behave | Application configuration | Result | | -------------------------------------- | ------------------------------------ | | No application configuration | No application-selected policies run | | `inboundPolicyChain` is absent or `[]` | No application-selected policies run | | `inboundPolicyChain` contains entries | Entries run in the listed order | | An entry has `enabled: false` | That entry is skipped | If a policy returns a response, later entries do not run. Successful 2xx responses still receive the budget check described below before they are sent. Other responses, including authentication failures, remain unchanged. If a chain is invalid, the request fails closed with an error that identifies the entry to fix. This inbound executor does not run `outboundPolicyChain`. That field is reserved for an outbound configuration executor on the response pipeline. ## Automatic budget enforcement The executor enforces budgets configured on the application, its parent teams, and the gateway. This does not require a Metering policy and applies even when the application's policy chain is empty. An application cannot override a shared budget with a higher limit of its own. Request logs indicate when automatic budget checks run, including when the application has no Metering policy. Warnings identify exceeded budgets, blocked requests, fallback decisions, and checks that could not be completed. When a budget blocks a provider request, the gateway uses the configured quota fallback model or returns `429 Too Many Requests`. Successful responses from policies, including Semantic Cache hits, are also subject to budgets. A blocked response returns `429 Too Many Requests` without using a fallback model. Warning-only budgets do not block responses. Authentication failures and other non-2xx policy responses are preserved. Place Metering before Semantic Cache to count cache hits toward request limits. Budget changes can take time to take effect, and concurrent requests can exceed a budget before further requests are blocked. If a budget check is unavailable, requests can proceed unless a budget has already been determined to block them. ## Build an AI Gateway from scratch ### 1. Declare the policies Add the optional configuration loader, the executor, and every policy an application may select to `config/policies.json`. Optionally declare AI Gateway Authentication when applications or routes will authenticate with application API keys. The following example allows applications to select model filtering and semantic caching. It does not assign either policy automatically; each application chooses the policies it needs in its `inboundPolicyChain`: ```json { "policies": [ { "name": "ai-gateway-auth-inbound", "policyType": "ai-gateway-auth", "handler": { "export": "AIGatewayAuthInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "cacheTtlSeconds": 60 } } }, { "name": "ai-gateway-configuration-loader-inbound", "policyType": "ai-gateway-configuration-loader", "handler": { "export": "AIGatewayConfigurationLoaderInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "cacheTtlSeconds": 60 } } }, { "name": "ai-gateway-configuration-executor-inbound", "policyType": "ai-gateway-configuration-executor", "handler": { "export": "AIGatewayConfigurationExecutorInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "cacheTtlSeconds": 60 } } }, { "name": "ai-gateway-metering-inbound", "policyType": "ai-gateway-metering", "handler": { "export": "AIGatewayMeteringInboundPolicy", "module": "$import(@zuplo/runtime)", "options": {} } }, { "name": "ai-gateway-model-filtering-inbound", "policyType": "ai-gateway-model-filtering", "handler": { "export": "AIGatewayModelFilteringInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "models": { "completions": { "allowList": ["openai/gpt-5-mini", "anthropic/claude-sonnet-4-6"] } } } } }, { "name": "ai-gateway-fallback-model-inbound", "policyType": "ai-gateway-fallback-model", "handler": { "export": "AIGatewayFallbackModelInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "models": { "completions": { "fallback": "anthropic/claude-sonnet-4-6" } }, "fallbackTimeoutSeconds": 60 } } }, { "name": "ai-gateway-semantic-cache-inbound", "policyType": "ai-gateway-semantic-cache", "handler": { "export": "AIGatewaySemanticCacheInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "semanticTolerance": 0.4, "expirationSecondsTtl": 3600 } } } ] } ``` Chain entries use declaration names, not policy types. The executor rejects undeclared policies and prevents direct or transitive re-entry into the configuration loader or executor. Use application and team policy templates to control which declared policies an application may edit or remove. ### 2. Add the loader and executor to the route Place the loader before the executor on each AI Gateway route. The loader loads configuration from the route's `app_id` path parameter (or reuses the channel when route-level auth already ran); the executor then runs that application's `inboundPolicyChain`. The AI Gateway handler runs after the selected inbound chain: ```json { "x-zuplo-route": { "corsPolicy": "none", "handler": { "export": "aiGatewayHandlerV2", "module": "$import(@zuplo/runtime)", "options": {} }, "policies": { "inbound": [ "ai-gateway-configuration-loader-inbound", "ai-gateway-configuration-executor-inbound" ] } } } ``` If you omit the loader, the executor loads the application configuration. Authentication is optional and placement controls its scope: - **App-level** — add `ai-gateway-auth-inbound` to an application's `inboundPolicyChain`. Only that application requires an API key. - **Route-level** — add `ai-gateway-auth-inbound` on the route **before** the loader (or before the executor on executor-only routes). That requires an API key for every application on the route. - **Internal applications** — for an application that is only called from inside the gateway with `context.invokeRoute`, add [Ensure Gateway Internal Invocation Only](/docs/policies/ai-gateway-internal-only-inbound) to its `inboundPolicyChain` instead of authentication. It rejects every request that arrives over the network and needs no API key. Use the same placement on each AI Gateway operation that should support application-selected chains. ### 3. Set an application's policy chain An application can inherit the options from `policies.json`. Include `ai-gateway-auth-inbound` when this application should require an API key: ```json { "inboundPolicyChain": [ { "name": "ai-gateway-auth-inbound" }, { "name": "ai-gateway-model-filtering-inbound" }, { "name": "ai-gateway-fallback-model-inbound" }, { "name": "ai-gateway-metering-inbound", "options": { "budgetRules": [ { "budgetBy": "app", "meters": [ { "meter": "requests", "period": "daily", "value": 1000, "action": "block" } ] } ] } }, { "name": "ai-gateway-semantic-cache-inbound" } ] } ``` Place authentication first when the app requires a key, then model filtering, fallback-model, and metering so metering can activate the resolved quota fallback. Put policies that may short-circuit, such as semantic cache, after metering so those requests still count toward request limits. An application can also provide a complete options object for an entry: ```json { "inboundPolicyChain": [ { "name": "ai-gateway-model-filtering-inbound", "options": { "models": { "completions": { "allowList": ["anthropic/claude-sonnet-4-6"] } } } } ] } ``` Entry options replace the declaration's entire options object; fields are not merged. Omit `options` to inherit the complete `handler.options` value from `policies.json`. An entry may also carry portal/template ACL metadata in `permissions`. The executor accepts this field and ignores it when running the chain; unknown keys under `permissions` fail closed: ```json { "inboundPolicyChain": [ { "name": "ai-gateway-model-filtering-inbound", "permissions": { "canEdit": true, "canRemove": false }, "options": { "models": { "completions": { "allowList": ["anthropic/claude-sonnet-4-6"] } } } } ] } ``` | `permissions` field | Meaning (portal/templates) | | ------------------- | ---------------------------------------------------- | | `canEdit` | Whether the application may edit the entry's options | | `canRemove` | Whether the application may remove the entry | Omit `permissions`, or either flag, when the portal does not need to record that constraint on the stored chain. ## Configure expression budget rules `budgetRules` in the application configuration can apply a budget to a value selected from the request. Each expression rule requires one or more meter rows: ```json { "budgetRules": [ { "budgetBy": "expression", "expression": "request.user.data[\"team-id\"]", "meters": [ { "meter": "requests", "period": "daily", "value": 10000, "action": "warn" }, { "meter": "requests", "period": "daily", "value": 12000, "action": "block" } ] } ] } ``` | Field | Supported values | | ---------- | ---------------------------------------------------- | | `budgetBy` | `expression` | | `meter` | `cost`, `requests`, `tokens` | | `period` | `hourly`, `daily`, `weekly`, `monthly` | | `action` | `warn` records a warning; `block` returns HTTP `429` | | `value` | A finite number greater than zero | A rule cannot repeat the same meter, period, and action. When a meter and period have both actions, the `warn` value must be less than the `block` value. The policy does not treat entries whose `budgetBy` is `app` as expression budget rules. ### Expressions An expression selects one value from the request or its context, such as `request.headers.get("x-user-id")` or `request.user.data["team-id"]`. Every distinct value gets its own budget. The expression grammar, the selectable properties, what a value must be, and what happens when an expression does not resolve are documented once, on the [AI Gateway Metering](/docs/policies/ai-gateway-metering-inbound) policy page. The same grammar applies here. Two things are specific to application configuration. A policy that sets a value an expression reads must run before Semantic Cache, which can answer without reaching the rest of the chain. And you can change an expression in the application configuration without rebuilding or redeploying the gateway. When writing raw JSON, escape the double quotes a bracket segment needs: ```json { "expression": "request.user.data[\"team-id\"]" } ``` After a JSON parser decodes that value the expression is `request.user.data["team-id"]`; the backslashes are not part of its identity. Code that builds application configuration should create an ordinary string and let its JSON serializer add the transport escaping. An expression selects one value and cannot combine several. To budget by a composite value, compute it in a custom policy earlier in the chain and store it somewhere an expression can read: ```ts import { ZuploContext, ZuploRequest } from "@zuplo/runtime"; export default function setBudgetDimension( request: ZuploRequest, context: ZuploContext, ): ZuploRequest { const headers = new Headers(request.headers); headers.set( "x-budget-customer", `${context.route.path}:${request.user?.sub ?? "anonymous"}`, ); return new ZuploRequest(request, { headers }); } ``` The corresponding expression is `request.headers.get("x-budget-customer")`. Configuration code should create that ordinary string and pass the enclosing object to `JSON.stringify`; the serializer adds the JSON transport escaping. Headers you set or remove in the inbound chain control what reaches your AI provider. For example, you can add a session header or remove an internal customer header before sending the request. Gateway-owned headers are excluded, including credentials, `Content-Type`, `Accept`, and headers starting with `zp-` or `cf-`; setting a credential header does not override your configured provider credential. That includes headers you add only for the gateway's own use. The `x-budget-customer` header above carries the caller's user id, and unless you remove it after the metering policy has read it, the provider receives it on every request. Add a `remove-headers-inbound` entry after the metering entry: ```json [ { "name": "set-budget-dimension" }, { "name": "ai-gateway-metering-inbound" }, { "name": "remove-headers-inbound", "options": { "headers": ["x-budget-customer"] } } ] ``` Declare `remove-headers-inbound` in `config/policies.json` so the chain can select it. Invalid budget rules and unsupported expressions are logged and skipped. Other valid rules continue to apply. ## Write a custom policy for the chain A chain entry can run any declared custom policy. The policy uses the standard inbound policy signature and receives its options from the chain entry (or the declaration, when the entry omits `options`). Entry-owned options are deep-copied for each invocation, so mutating them is safe. Inherited declaration options are passed through unchanged to match static route behavior; do not mutate them. ```ts import { AIGatewayModelRouting, AIGatewayModels, ZuploContext, ZuploRequest, } from "@zuplo/runtime"; interface CheapestModelOptions { providers?: string[]; } export default async function cheapestModel( request: ZuploRequest, context: ZuploContext, options: CheapestModelOptions, ): Promise { // The authenticated application: sub is the application name, data its // metadata. context.log.info(`Routing for application ${request.user?.sub}`); // Read the live provider and model catalog. Credentials are never returned. const allowed = options.providers ?? ["openai", "anthropic"]; const candidates = (await AIGatewayModels.load(context)) .filter(({ providerName }) => allowed.includes(providerName.toLowerCase())) .flatMap((provider) => provider.models.map((model) => ({ providerName: provider.providerName, model, })), ) .filter( ({ model }) => model.capability === "completions" && model.status === "active", ) .sort( (left, right) => left.model.inputCostPerToken + left.model.outputCostPerToken - (right.model.inputCostPerToken + right.model.outputCostPerToken), ); const cheapest = candidates[0]; if (!cheapest) { // Returning a Response short-circuits the chain and answers the request. return new Response("No eligible model", { status: 503 }); } // Select the model the gateway calls for this request. await AIGatewayModelRouting.set(context, { completions: `${cheapest.providerName}/${cheapest.model.model}`, }); return request; } ``` From a chain policy you can: - **Read the calling application** from `request.user`: `sub` is the application name and `data` its metadata. - **Read the model catalog** with `AIGatewayModels.load(context)`. - **Choose the model for the request** with `AIGatewayModelRouting.set(context, routing)`, and read the current selection with `AIGatewayModelRouting.get(context)`. - **Block or answer the request** by returning a `Response`; later chain entries do not run. - **Receive settings** through the chain entry's `options`, exactly like the built-in policies. ## Store secrets safely Application chain options can reference project environment variables. Use a standalone reference when the complete option value comes from one variable, or embed one or more references in a larger string: ```json { "inboundPolicyChain": [ { "name": "my-ai-guardrail", "options": { "apiKey": "$env(MY_GUARDRAIL_API_KEY)", "authorization": "Bearer $env(MY_GUARDRAIL_API_KEY)" } } ] } ``` The executor resolves `$env(NAME)` references when it instantiates an enabled chain entry. Resolution visits string values recursively through nested objects and arrays. Object keys remain literal. If a referenced variable is missing or restricted, the executor rejects the configuration with an error that identifies the variable and the chain entry; it never substitutes an empty string. To share the same environment-backed options across applications, keep them in the pre-declared policy's `handler.options`, then omit `options` from the application chain entry to inherit them: ```json { "name": "my-ai-guardrail", "policyType": "custom-code-inbound", "handler": { "export": "default", "module": "$import(./modules/my-ai-guardrail)", "options": { "apiKey": "$env(MY_GUARDRAIL_API_KEY)" } } } ``` ```json { "inboundPolicyChain": [ { "name": "my-ai-guardrail" } ] } ``` Declared `handler.options` use the normal build-time environment resolution. Application entry options are resolved at runtime and replace the declaration's complete options object; the two objects are not merged. ## Chain entry reference Every chain entry supports: - `name` (required): Name of a declaration in `policies.json`. - `options` (optional): Complete replacement options for this invocation. Omit it to inherit the declaration's options. - `enabled` (optional): Set to `false` to keep an entry in the configuration without running it. Omitted or `true` entries run normally. Disabled entries are still validated. They must be well-formed, name a declared policy, and cannot select the configuration loader or executor. Their options are not instantiated, so their environment references are not resolved. The executor permits repeated entries and cannot infer the behavior of custom or wrapper policies. Configuration authors are responsible for avoiding repeated invocation when a policy is not safe to run more than once. ## Empty chains An application without an `inboundPolicyChain`, or with an explicit empty array, runs no application-selected policies: ```json { "inboundPolicyChain": [] } ``` Ensure the loaded configuration supplies every policy required for the request, or attach required policies directly to the route. ## Configuration checklist Before deploying: 1. Declare every selectable policy in `config/policies.json`. 2. Configure application and team policy templates with the policies each app may edit or remove. 3. Put shared environment-backed values in declared policy options, or use `$env(...)` in application entry option values when each application needs its own complete options object. 4. Place the configuration loader before the executor on every AI Gateway route (or the executor alone when you prefer the combined path). Add AI Gateway Authentication to an application's chain for that app only, or before the loader on the route to require keys for every app. 5. Ensure every application chain includes the policies required for that route. Read more about [how policies work](/articles/policies)