# AI Gateway Model Filtering (v2) Policy AI Gateway Model Filtering (v2) controls which `providerName/model` references clients may select. Use an `allowList` for a curated catalog with a default, or a `blockList` for an open catalog with explicit exclusions. Put this policy before AI Gateway Fallback Model (v2), which adds fallback behavior only after a primary model has passed filtering. :::caution{title="Beta"} This policy is in beta. You can use it today, but it may change in non-backward compatible ways before the final release. ::: ## Configuration The configuration shows how to configure the policy in the 'policies.json' document. ```json title="config/policies.json" { "name": "my-ai-gateway-model-filtering-v2-inbound-policy", "policyType": "ai-gateway-model-filtering-v2-inbound", "handler": { "export": "AIGatewayModelFilteringV2InboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "models": {} } } } ``` ### Policy Configuration - `name` <string> - The name of your policy instance. This is used as a reference in your routes. - `policyType` <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be `ai-gateway-model-filtering-v2-inbound`. - `handler.export` <string> - The name of the exported type. Value should be `AIGatewayModelFilteringV2InboundPolicy`. - `handler.module` <string> - The module containing the policy. Value should be `$import(@zuplo/runtime)`. - `handler.options` <object> - The options for this policy. [See Policy Options](#policy-options) below. ### Policy Options The options for this policy are specified below. All properties are optional unless specifically marked as required. - `models` **(required)** <object> - Model filtering rules grouped by AI Gateway capability. - `completions` <undefined> - Rules for chat completions, Responses, and Anthropic Messages requests. - `embeddings` <undefined> - Rules for embedding requests. ## Using the Policy # AI Gateway Model Filtering (v2) Policy Use this policy when an AI Gateway route must restrict which models clients may select. The policy is optional. Choose the setup that matches the route: - Omit Model Filtering to let each request select any available model. Every request must provide `model` as `providerName/model`. - Use an `allowList` to expose a curated set of models and provide a default model. - Use a `blockList` to permit available models except for specific exclusions. - Use a custom routing policy when model selection depends on request data or application logic that an allow list or block list cannot express. `providerName` is the Provider Name configured in the Zuplo Portal. The text after the first slash is the provider-specific model ID, so model IDs may contain additional slashes. ## Routing without Model Filtering When neither Model Filtering nor a custom routing policy selects a model, the AI Gateway handler reads the request body's `model` and uses it as the primary routing target: ```json { "model": "openai/gpt-4o-mini", "messages": [{ "role": "user", "content": "Hello" }] } ``` The handler validates that: - `model` is a string in `providerName/model` form; - the Provider Name is configured; - the model is available for the route's capability; - the Provider Assignment has usable credentials. Missing, bare, non-string, or malformed model values receive an OpenAI-compatible 400 `invalid_request_error`. A validly formatted target that cannot be fulfilled is reported as a routing configuration error with a suggested fix. When Semantic Cache is attached, a cache hit also validates request-derived routing before returning the cached response. A removed, deprecated, or otherwise unavailable target therefore follows the same routing configuration error path instead of bypassing validation with a cached response. Native route requirements are checked too, so an Anthropic Messages cache hit cannot be served by an OpenAI-backed Provider Name. Attach Model Filtering only when the gateway must enforce model-selection rules. An attached policy must always have a valid, non-empty `models` configuration. ### Responses management operations These Responses API operations do not have a request body: - `GET /v1/responses/:responseId` - `GET /v1/responses/:responseId/input_items` - `DELETE /v1/responses/:responseId` Because they cannot supply `model`, they require routing to be selected before the handler runs. Configure a `completions.allowList` in Model Filtering so its first entry supplies the default, or use a custom inbound policy that calls `AIGatewayModelRouting.set(context, { completions: "providerName/model" })`. Without preselected routing, the handler returns an OpenAI-compatible 400 `invalid_request_error` with this configuration guidance. ## Policy order Place Model Filtering before Fallback Model in the inbound policy chain: ```text Model Filtering -> Fallback Model -> AI Gateway handler ``` Model Filtering accepts or rejects the request and creates the primary model selection. Fallback Model can then enrich that allowed selection without bypassing the filter. Fallback Model does not create a primary selection by itself. ## Options `models` must contain `completions`, `embeddings`, or both. Each capability chooses exactly one mode: - `allowList` creates a curated catalog. Only listed models are accepted, and the first entry is used when a request omits `model`. - `blockList` leaves the catalog open except for named models. Every request must include `model`. Each configured list must contain at least one entry. A capability cannot define both `allowList` and `blockList`, and unsupported fields are rejected. Every list entry is a plain `providerName/model` string. Matching is case-insensitive, while configured casing is preserved for the upstream request. Route-target objects and fallback fields belong in the Fallback Model policy. Configure every capability served by a route using this policy. A route using `/v1/embeddings` needs `embeddings`; Chat Completions, Responses, and Anthropic Messages routes need `completions`. If the policy is attached but the active capability has no rules, the request receives a 403 response explaining which capability to add. ## Allow-list example ```json { "name": "ai-gateway-model-filtering-v2-inbound", "policyType": "ai-gateway-model-filtering-v2", "handler": { "export": "AIGatewayModelFilteringV2InboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "models": { "completions": { "allowList": ["openai/gpt-4o-mini", "anthropic/claude-haiku-4-5"] }, "embeddings": { "allowList": ["openai/text-embedding-3-small"] } } } } } ``` ## Block-list example ```json { "models": { "completions": { "blockList": ["openai/deprecated-model", "anthropic/deprecated-model"] } } } ``` In block-list mode, an unknown Provider Name is rejected even if the model is not listed. Entries whose Provider Names are absent from the live catalog are reported as warnings because they cannot match a request. ## Request behavior | Situation | Result | | --------------------------------------------------- | ------------------------------------------------------ | | Allow list, request omits `model` | The first allow-list entry is selected. | | Allow list, request names a listed model | The matching configured entry is selected. | | Allow list, request names an unlisted model | 403 response listing the allowed models. | | Block list, request omits `model` | 400 response asking for `providerName/model`. | | Block list, request names a blocked model | 403 response. | | Either mode, request uses a bare or malformed model | 400 response explaining the required format. | | Policy has no rules for the route capability | 403 response explaining which capability to configure. | | Another inbound policy already selected routing | Model Filtering leaves that selection unchanged. | Native routes also enforce provider type. `/v1/responses` requires a Provider Name backed by OpenAI, and `/v1/messages` requires a Provider Name backed by Anthropic. Provider Names may be custom labels; validation uses the provider type configured for that Provider Assignment. For example, this embedding request is evaluated against `models.embeddings`: ```json { "model": "openai/text-embedding-3-small", "input": "Search text" } ``` ## Adding fallbacks Declare AI Gateway Fallback Model (v2) separately and place it immediately after this policy. Its `fallback` handles retryable errors and timeouts; `quotaFallback` handles usage-limit signals independently. ## Write your own routing policy Everything Model Filtering does is built on two public primitives, so a custom inbound policy can replace it entirely. The policy's job is to choose one allowed target and store it: - `AIGatewayModels.load(context)` returns the cached provider catalog, including each model's capability, status, and per-token pricing. - `AIGatewayModelRouting.set(context, routing)` validates the routing, resolves provider credentials internally, and stores the selection that the AI Gateway handler consumes. ```typescript import { AIGatewayModelRouting, AIGatewayModels, type ZuploContext, type ZuploRequest, } from "@zuplo/runtime"; export default async function selectModel( request: ZuploRequest, context: ZuploContext, ) { const providers = await AIGatewayModels.load(context); const openAI = providers.find( (provider) => provider.providerName === "openai", ); const model = openAI?.models.find( (candidate) => candidate.capability === "completions" && candidate.status === "active", ); if (!model) { throw new Error("No active OpenAI completions model is available"); } await AIGatewayModelRouting.set(context, { completions: `openai/${model.model}`, }); return request; } ``` Attach the module as an ordinary inbound policy instead of Model Filtering: ```json { "name": "my-model-filtering-inbound", "policyType": "custom-code-inbound", "handler": { "export": "default", "module": "$import(./modules/my-model-filtering)" } } ``` `AIGatewayModelRouting.get(context)` returns the sanitized, normalized routing for the current request and never returns credentials. The AI Gateway handler consumes a custom selection even when Model Filtering is not attached. If neither kind of policy creates a selection, the handler derives one from the request's required `providerName/model`. Policy order determines precedence: 1. Routing selected before Model Filtering remains authoritative because Model Filtering leaves an existing selection unchanged. 2. Model Filtering creates routing when no earlier policy selected it. 3. A custom policy placed after Model Filtering may deliberately replace that selection. 4. If no policy selects routing, the handler derives it from the request. Prefer one policy as the primary selector so the route's intent is easy to understand. Read more about [how policies work](/articles/policies)