Copy page

Configuration

AI Gateway Configuration Executor Policy

The AI Gateway Configuration Executor loads each application's configuration (when auth or the configuration loader has not already) and runs an ordered inbound policy chain assembled from policies declared by the gateway. Applications may also include authentication in their own chain. This makes it possible to offer different routing, caching, guardrail, metering, and tracing behavior from one AI Gateway deployment.

Prefer placing ai-gateway-configuration-loader-v2-inbound before this executor on the route so loading and chain execution stay separate. When the loader is omitted, this executor still loads configuration itself.

The gateway owner decides which policy declarations are available, while application and team templates control which entries an application may edit or remove. Each request carries the resulting chain and policy options.

Configuration

The configuration shows how to configure the policy in the 'policies.json' document.

Code
{
  "name": "my-ai-gateway-configuration-executor-v2-inbound-policy",
  "policyType": "ai-gateway-configuration-executor-v2-inbound",
  "handler": {
    "export": "AIGatewayConfigurationExecutorV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "cacheTtlSeconds": 10
    }
  }
}

Policy Configuration

• name <string> - The name of your policy instance. This is used as a reference in your routes.
• policyType <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be ai-gateway-configuration-executor-v2-inbound.
• handler.export <string> - The name of the exported type. Value should be AIGatewayConfigurationExecutorV2InboundPolicy.
• handler.module <string> - The module containing the policy. Value should be $import(@zuplo/runtime).
• handler.options <object> - The options for this policy. See Policy Options below.

Policy Options

The options for this policy are specified below. All properties are optional unless specifically marked as required.

• cacheTtlSeconds <number> - The time in seconds to cache app configurations loaded by app_id. Defaults to 10 seconds when omitted. Higher values decrease latency; lower values pick up portal changes sooner. Cached results remain valid until the cache expires even if the configuration changes in the portal. This cache is only used when neither ai-gateway-auth-v2-inbound nor ai-gateway-configuration-loader-v2-inbound already loaded the configuration for the request. Defaults to 10.

Using the Policy

AI Gateway Configuration Executor

The AI Gateway Configuration Executor loads each application's configuration (when auth or ai-gateway-configuration-loader-v2-inbound has not already) and runs its ordered inbound policy chain. Configuration comes from route-level ai-gateway-auth-v2-inbound or the configuration loader when either already ran, otherwise from the route's app_id path parameter via Gateway Service. Applications can still require API keys later by including ai-gateway-auth-v2-inbound in their own inboundPolicyChain.

Prefer placing the dedicated configuration loader before this executor on the route. When the loader is omitted, this executor still loads configuration itself before running the chain.

The gateway owner remains in control:

• Every selectable policy must be declared in config/policies.json.
• Application and team policy templates determine which entries applications may edit or remove before the resulting chain reaches the gateway.

How application chains behave

Application configuration	Result
No application configuration	No application-selected policies run
inboundPolicyChain is absent or []	No application-selected policies run
inboundPolicyChain contains entries	Entries run in the listed order
An entry has enabled: false	That entry is skipped

If a policy returns a response, that response is sent immediately and later entries do not run. If a chain is invalid, the request fails closed with an error that identifies the entry to fix.

This inbound executor does not run outboundPolicyChain. That field is reserved for an outbound configuration executor on the response pipeline.

Build an AI Gateway from scratch

1. Declare the policies

Add the optional configuration loader, the executor, and every policy an application may select to config/policies.json. Optionally declare AI Gateway Authentication when applications or routes will authenticate with application API keys.

The following example allows applications to select model filtering and semantic caching. It does not assign either policy automatically; each application chooses the policies it needs in its inboundPolicyChain:

Code
{
  "policies": [
    {
      "name": "ai-gateway-auth-v2-inbound",
      "policyType": "ai-gateway-auth-v2",
      "handler": {
        "export": "AIGatewayAuthV2InboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "cacheTtlSeconds": 60
        }
      }
    },
    {
      "name": "ai-gateway-configuration-loader-v2-inbound",
      "policyType": "ai-gateway-configuration-loader-v2",
      "handler": {
        "export": "AIGatewayConfigurationLoaderV2InboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "cacheTtlSeconds": 60
        }
      }
    },
    {
      "name": "ai-gateway-configuration-executor-v2-inbound",
      "policyType": "ai-gateway-configuration-executor-v2",
      "handler": {
        "export": "AIGatewayConfigurationExecutorV2InboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "cacheTtlSeconds": 60
        }
      }
    },
    {
      "name": "ai-gateway-metering-v2-inbound",
      "policyType": "ai-gateway-metering-v2",
      "handler": {
        "export": "AIGatewayMeteringV2InboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {}
      }
    },
    {
      "name": "ai-gateway-model-filtering-v2-inbound",
      "policyType": "ai-gateway-model-filtering-v2",
      "handler": {
        "export": "AIGatewayModelFilteringV2InboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "models": {
            "completions": {
              "allowList": ["openai/gpt-5-mini", "anthropic/claude-sonnet-4-6"]
            }
          }
        }
      }
    },
    {
      "name": "ai-gateway-fallback-model-v2-inbound",
      "policyType": "ai-gateway-fallback-model-v2",
      "handler": {
        "export": "AIGatewayFallbackModelV2InboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "models": {
            "completions": {
              "fallback": "anthropic/claude-sonnet-4-6"
            }
          },
          "fallbackTimeoutSeconds": 60
        }
      }
    },
    {
      "name": "ai-gateway-semantic-cache-v2-inbound",
      "policyType": "ai-gateway-semantic-cache-v2",
      "handler": {
        "export": "AIGatewaySemanticCacheV2InboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "semanticTolerance": 0.4,
          "expirationSecondsTtl": 3600
        }
      }
    }
  ]
}

Chain entries use declaration names, not policy types. The executor rejects undeclared policies and prevents direct or transitive re-entry into the configuration loader or executor. Use application and team policy templates to control which declared policies an application may edit or remove.

2. Add the loader and executor to the route

Place the loader before the executor on each AI Gateway route. The loader loads configuration from the route's app_id path parameter (or reuses the channel when route-level auth already ran); the executor then runs that application's inboundPolicyChain. The AI Gateway handler runs after the selected inbound chain:

Code
{
  "x-zuplo-route": {
    "corsPolicy": "none",
    "handler": {
      "export": "aiGatewayHandlerV2",
      "module": "$import(@zuplo/runtime)",
      "options": {}
    },
    "policies": {
      "inbound": [
        "ai-gateway-configuration-loader-v2-inbound",
        "ai-gateway-configuration-executor-v2-inbound"
      ]
    }
  }
}

Routes that list only the executor keep working — the executor loads configuration when the channel is empty.

Authentication is optional and placement controls its scope:

• App-level — add ai-gateway-auth-v2-inbound to an application's inboundPolicyChain. Only that application requires an API key.
• Route-level — add ai-gateway-auth-v2-inbound on the route before the loader (or before the executor on executor-only routes). That requires an API key for every application on the route.

Use the same placement on each AI Gateway operation that should support application-selected chains.

3. Set an application's policy chain

An application can inherit the options from policies.json. Include ai-gateway-auth-v2-inbound when this application should require an API key:

Code
{
  "inboundPolicyChain": [
    {
      "name": "ai-gateway-auth-v2-inbound"
    },
    {
      "name": "ai-gateway-model-filtering-v2-inbound"
    },
    {
      "name": "ai-gateway-fallback-model-v2-inbound"
    },
    {
      "name": "ai-gateway-metering-v2-inbound",
      "options": {
        "limits": {
          "requests": {
            "daily": {
              "enabled": true,
              "limit": 1000
            }
          }
        }
      }
    },
    {
      "name": "ai-gateway-semantic-cache-v2-inbound"
    }
  ]
}

Place authentication first when the app requires a key, then model filtering, fallback-model, and metering so metering can activate the resolved quota fallback. Put policies that may short-circuit, such as semantic cache, after metering so those requests still count toward request limits.

An application can also provide a complete options object for an entry:

Code
{
  "inboundPolicyChain": [
    {
      "name": "ai-gateway-model-filtering-v2-inbound",
      "options": {
        "models": {
          "completions": {
            "allowList": ["anthropic/claude-sonnet-4-6"]
          }
        }
      }
    }
  ]
}

Entry options replace the declaration's entire options object; fields are not merged. Omit options to inherit the complete handler.options value from policies.json.

An entry may also carry portal/template ACL metadata in permissions. The executor accepts this field and ignores it when running the chain; unknown keys under permissions fail closed:

Code
{
  "inboundPolicyChain": [
    {
      "name": "ai-gateway-model-filtering-v2-inbound",
      "permissions": {
        "canEdit": true,
        "canRemove": false
      },
      "options": {
        "models": {
          "completions": {
            "allowList": ["anthropic/claude-sonnet-4-6"]
          }
        }
      }
    }
  ]
}

permissions field	Meaning (portal/templates)
canEdit	Whether the application may edit the entry's options
canRemove	Whether the application may remove the entry

Omit permissions, or either flag, when the portal does not need to record that constraint on the stored chain.

Configure expression budget rules

budgetRules in the application configuration can apply a budget to a value selected from the request. Each expression rule requires one or more meter rows:

Code
{
  "budgetRules": [
    {
      "budgetBy": "expression",
      "expression": "request.user.data[\"team-id\"]",
      "meters": [
        {
          "meter": "requests",
          "period": "daily",
          "value": 10000,
          "action": "warn"
        },
        {
          "meter": "requests",
          "period": "daily",
          "value": 12000,
          "action": "block"
        }
      ]
    }
  ]
}

Field	Supported values
budgetBy	expression
meter	cost, requests, tokens
period	hourly, daily, weekly, monthly
action	warn records a warning; block returns HTTP 429
value	A finite number greater than zero

A rule cannot repeat the same meter, period, and action. When a meter and period have both actions, the warn value must be less than the block value. The policy does not treat entries whose budgetBy is app as expression budget rules.

Supported expressions

Expressions read one string or safe integer from request-time data. Use these canonical forms when writing application configuration:

Source	Canonical expression example
Request URL or method	request.url, request.method
One request header	request.headers.get("x-customer-id")
Authenticated subject	request.user.sub
User data	request.user.data.profile.organizationId
Query parameter	request.query.customerId
Query parameter (map form)	request.searchParams.customerId
Route parameter	request.params.productId
Custom context data	context.custom.account.id
Route data	context.route.path
Incoming request data	context.incomingRequestProperties.country
Request or context identity	request.user.sub, context.requestId
A key requiring brackets	request.user.data["team-id"]

The expression field contains the complete expression. Do not wrap it in an interpolation marker such as ${...}. The runtime resolves it for every request after the application's inbound policy chain finishes. Updating an application's configuration changes which value later requests select without rebuilding or redeploying the gateway.

JSON encoding and expression syntax are separate. When writing raw JSON, escape the double quotes required by a bracket segment:

Code
{
  "expression": "request.user.data[\"team-id\"]"
}

After a JSON parser decodes this value, the expression is request.user.data["team-id"]; the backslashes are not part of its identity. Code that creates application configuration should build an ordinary string and let its JSON serializer handle the transport escaping:

Code
const expression = 'request.user.data["team-id"]';
const body = JSON.stringify({ expression });

Header expressions are terminal. Header names are case-insensitive; use lowercase in stored expressions. The parser accepts the canonical request.headers.get("content-type") form and the equivalent request.headers.content-type and request.headers["content-type"] forms. The canonical form keeps header access visually distinct from an ordinary object property.

A dot property starts with an ASCII letter, _, or $, followed by those characters, ASCII digits, or hyphens. Canonical rendering uses quoted brackets for keys containing $ or -, or any other non-identifier character. For example, context.custom.team-id is accepted and its canonical form is context.custom["team-id"]. Single- and double-quoted bracket properties are accepted. Whitespace immediately inside brackets is accepted but is not canonical.

The selectable data model is:

Root	Selectable properties
request	url, method, headers, user.sub, user.data, query, searchParams, params, and the scalar Fetch request metadata bodyUsed, cache, credentials, destination, integrity, keepalive, mode, redirect, referrer, and referrerPolicy
context	contextId, requestId, custom, route, and incomingRequestProperties

request.body, context.log, policy invocation methods, and every other host object are outside the selectable data model. The evaluator receives a plain-data snapshot of the selected root. It never receives the live request or context object.

Expressions have these limits:

• The expression is at most 1,024 UTF-8 bytes and contains well-formed Unicode.
• Quoted keys are non-empty and do not contain double quotes, backslashes, control characters, U+2028, or U+2029. String escapes are not supported.
• Canonical expressions contain no syntax whitespace. Spaces inside a quoted key are part of the key.
• Wildcards, recursive descent, filters, numeric bracket indexes, calls other than the terminal header .get("name"), function arguments, and roots other than request and context are not supported. For example, request.url.substring(5, 1) is not a supported expression.
• Traversal reads own properties from plain objects and keys from declared Maps. It does not traverse arrays, class instances, inherited properties, or the property names __proto__, constructor, and prototype.

Expressions observe values after the application's inbound policy chain finishes. An authentication policy can populate request.user. A custom policy earlier in the chain can derive a value and place it in a request header:

Code
import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default function setBudgetDimension(
  request: ZuploRequest,
  context: ZuploContext,
): ZuploRequest {
  const headers = new Headers(request.headers);
  headers.set(
    "x-budget-customer",
    `${context.route.path}:${request.user?.sub ?? "anonymous"}`,
  );
  return new ZuploRequest(request, { headers });
}

The corresponding expression is request.headers.get("x-budget-customer"). Configuration code should create that ordinary string and pass the enclosing object to JSON.stringify; the serializer adds the JSON transport escaping.

The expression chooses one value; it does not concatenate values, run JavaScript, or evaluate conditions. Compute composite values in an earlier policy and store the result in a supported request or context property.

The selected value must be a string or a safe integer. The runtime converts the value to well-formed NFC Unicode and ignores it when it exceeds 256 UTF-8 bytes or contains control characters, U+2028, or U+2029. A missing or invalid value does not contribute to the rule for that request.

The exact stored expression is the rule identity and analytics dimension name. Store its authored bytes unchanged. Different accepted spellings, such as single quotes and double quotes, identify different rules even when they select the same value.

Malformed budget rules are logged and ignored individually. An unsupported expression is also logged and omitted from dimension capture and quota tokens; other valid rules continue to run.

Write a custom policy for the chain

A chain entry can run any declared custom policy. The policy uses the standard inbound policy signature and receives its options from the chain entry (or the declaration, when the entry omits options). Entry-owned options are deep-copied for each invocation, so mutating them is safe. Inherited declaration options are passed through unchanged to match static route behavior; do not mutate them.

Code
import {
  AIGatewayModelRouting,
  AIGatewayModels,
  ZuploContext,
  ZuploRequest,
} from "@zuplo/runtime";

interface CheapestModelOptions {
  providers?: string[];
}

export default async function cheapestModel(
  request: ZuploRequest,
  context: ZuploContext,
  options: CheapestModelOptions,
): Promise<ZuploRequest | Response> {
  // The authenticated application: sub is the application name, data its
  // metadata.
  context.log.info(`Routing for application ${request.user?.sub}`);

  // Read the live provider and model catalog. Credentials are never returned.
  const allowed = options.providers ?? ["openai", "anthropic"];
  const candidates = (await AIGatewayModels.load(context))
    .filter(({ providerName }) => allowed.includes(providerName.toLowerCase()))
    .flatMap((provider) =>
      provider.models.map((model) => ({
        providerName: provider.providerName,
        model,
      })),
    )
    .filter(
      ({ model }) =>
        model.capability === "completions" && model.status === "active",
    )
    .sort(
      (left, right) =>
        left.model.inputCostPerToken +
        left.model.outputCostPerToken -
        (right.model.inputCostPerToken + right.model.outputCostPerToken),
    );

  const cheapest = candidates[0];
  if (!cheapest) {
    // Returning a Response short-circuits the chain and answers the request.
    return new Response("No eligible model", { status: 503 });
  }

  // Select the model the gateway calls for this request.
  await AIGatewayModelRouting.set(context, {
    completions: `${cheapest.providerName}/${cheapest.model.model}`,
  });
  return request;
}

From a chain policy you can:

• Read the calling application from request.user: sub is the application name and data its metadata.
• Read the model catalog with AIGatewayModels.load(context).
• Choose the model for the request with AIGatewayModelRouting.set(context, routing), and read the current selection with AIGatewayModelRouting.get(context).
• Block or answer the request by returning a Response; later chain entries do not run.
• Receive settings through the chain entry's options, exactly like the built-in policies.

Store secrets safely

Application chain options can reference project environment variables. Use a standalone reference when the complete option value comes from one variable, or embed one or more references in a larger string:

Code
{
  "inboundPolicyChain": [
    {
      "name": "my-ai-guardrail",
      "options": {
        "apiKey": "$env(MY_GUARDRAIL_API_KEY)",
        "authorization": "Bearer $env(MY_GUARDRAIL_API_KEY)"
      }
    }
  ]
}

The executor resolves $env(NAME) references when it instantiates an enabled chain entry. Resolution visits string values recursively through nested objects and arrays. Object keys remain literal. If a referenced variable is missing or restricted, the executor rejects the configuration with an error that identifies the variable and the chain entry; it never substitutes an empty string.

To share the same environment-backed options across applications, keep them in the pre-declared policy's handler.options, then omit options from the application chain entry to inherit them:

Code
{
  "name": "my-ai-guardrail",
  "policyType": "custom-code-inbound",
  "handler": {
    "export": "default",
    "module": "$import(./modules/my-ai-guardrail)",
    "options": {
      "apiKey": "$env(MY_GUARDRAIL_API_KEY)"
    }
  }
}

Code
{
  "inboundPolicyChain": [
    {
      "name": "my-ai-guardrail"
    }
  ]
}

Declared handler.options use the normal build-time environment resolution. Application entry options are resolved at runtime and replace the declaration's complete options object; the two objects are not merged.

Chain entry reference

Every chain entry supports:

• name (required): Name of a declaration in policies.json.
• options (optional): Complete replacement options for this invocation. Omit it to inherit the declaration's options.
• enabled (optional): Set to false to keep an entry in the configuration without running it. Omitted or true entries run normally.

Disabled entries are still validated. They must be well-formed, name a declared policy, and cannot select the configuration loader or executor. Their options are not instantiated, so their environment references are not resolved.

The executor permits repeated entries and cannot infer the behavior of custom or wrapper policies. Configuration authors are responsible for avoiding repeated invocation when a policy is not safe to run more than once.

Empty chains

An application without an inboundPolicyChain, or with an explicit empty array, runs no application-selected policies:

Code
{
  "inboundPolicyChain": []
}

Ensure the loaded configuration supplies every policy required for the request, or attach required policies directly to the route.

Configuration checklist

Before deploying:

1. Declare every selectable policy in config/policies.json.
2. Configure application and team policy templates with the policies each app may edit or remove.
3. Put shared environment-backed values in declared policy options, or use $env(...) in application entry option values when each application needs its own complete options object.
4. Place the configuration loader before the executor on every AI Gateway route (or the executor alone when you prefer the combined path). Add AI Gateway Authentication to an application's chain for that app only, or before the loader on the route to require keys for every app.
5. Ensure every application chain includes the policies required for that route.

Read more about how policies work