# AI Gateway Model Filtering Policy :::note{title="AI Gateway Policy"} This policy is for use with the [AI Gateway](/docs/ai-gateway/overview). See the AI Gateway documentation to learn how to configure and govern AI models with Zuplo. ::: AI Gateway Model Filtering controls which `providerName/model` references clients may select. Use an `allowList` for a curated catalog with a default, or a `blockList` for an open catalog with explicit exclusions. Put this policy before AI Gateway Fallback Model, which adds fallback behavior only after a primary model has passed filtering. ## Configuration The configuration shows how to configure the policy in the 'policies.json' document. ```json title="config/policies.json" { "name": "my-ai-gateway-model-filtering-inbound-policy", "policyType": "ai-gateway-model-filtering-inbound", "handler": { "export": "AIGatewayModelFilteringInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "models": { "completions": { "allowList": ["openai/gpt-4o-mini"] } } } } } ``` ### Policy Configuration - `name` <string> - The name of your policy instance. This is used as a reference in your routes. - `policyType` <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be `ai-gateway-model-filtering-inbound`. - `handler.export` <string> - The name of the exported type. Value should be `AIGatewayModelFilteringInboundPolicy`. - `handler.module` <string> - The module containing the policy. Value should be `$import(@zuplo/runtime)`. - `handler.options` <object> - The options for this policy. [See Policy Options](#policy-options) below. ### Policy Options The options for this policy are specified below. All properties are optional unless specifically marked as required. - `models` **(required)** <object> - Model filtering rules grouped by AI Gateway capability. - `completions` <undefined> - Rules for chat completions, Responses, and Anthropic Messages requests. - `embeddings` <undefined> - Rules for embedding requests. ## Using the Policy Use this policy when an AI Gateway route must restrict which models clients may select. Without it, an application can use any available model from the providers configured for the Zuplo project. The policy is optional. Choose the setup that matches the route: - Omit Model Filtering to let each request select any available model. Every request must provide `model` as `providerName/model`. - Use an `allowList` to expose a curated set of models and provide a default model. - Use a `blockList` to permit available models except for specific exclusions. - Use a custom routing policy when model selection depends on request data or application logic that an allow list or block list cannot express. `providerName` is the Provider Name configured in the Zuplo Portal. The text after the first slash is the provider-specific model ID, so model IDs may contain additional slashes. ## Routing without Model Filtering When neither Model Filtering nor a custom routing policy selects a model, the AI Gateway handler reads the request body's `model` and uses it as the primary routing target: ```json { "model": "openai/gpt-4o-mini", "messages": [{ "role": "user", "content": "Hello" }] } ``` The handler validates that: - `model` is a string in `providerName/model` form; - the Provider Name is configured; - the model is available for the route's capability; - the Provider Assignment has usable credentials. Missing, bare, non-string, or malformed model values receive an OpenAI-compatible 400 `invalid_request_error`. A validly formatted target that cannot be fulfilled is reported as a routing configuration error with a suggested fix. When Semantic Cache is attached, a cache hit also validates request-derived routing before returning the cached response. A removed, deprecated, or otherwise unavailable target therefore follows the same routing configuration error path instead of bypassing validation with a cached response. Native route requirements are checked too, so an Anthropic Messages cache hit cannot be served by an OpenAI-backed Provider Name. Attach Model Filtering only when the gateway must enforce model-selection rules. An attached policy must always have a valid, non-empty `models` configuration. ### Responses management operations These Responses API operations do not have a request body: - `GET /v1/responses/:responseId` - `GET /v1/responses/:responseId/input_items` - `DELETE /v1/responses/:responseId` Because they cannot supply `model`, they require routing to be selected before the handler runs. Attach AI Gateway Model Override with a `models.completions.default` model, configure a `completions.allowList` in Model Filtering so its first entry supplies the default, or use a custom inbound policy that calls `AIGatewayModelRouting.set(context, { completions: "providerName/model" })`. Without preselected routing, the handler returns an OpenAI-compatible 400 `invalid_request_error` with this configuration guidance. ## Policy order Place Model Filtering before Fallback Model in the inbound policy chain: ```text Model Filtering -> Fallback Model -> AI Gateway handler ``` Model Filtering accepts or rejects the request and creates the primary model selection. Fallback Model can then enrich that allowed selection without bypassing the filter. Fallback Model does not create a primary selection by itself. ## Options `models` must contain `completions`, `embeddings`, or both. Each capability chooses exactly one mode: - `allowList` creates a curated catalog. Only listed models are accepted, and the first entry is used when a request omits `model`. - `blockList` leaves the catalog open except for named models. Every request must include `model`. Each configured list must contain at least one entry. A capability cannot define both `allowList` and `blockList`, and unsupported fields are rejected. Every list entry is a plain `providerName/model` string. Matching is case-insensitive, while configured casing is preserved for the upstream request. Route-target objects and fallback fields belong in the Fallback Model policy. Configure every capability served by a route using this policy. A route using `/v1/embeddings` needs `embeddings`; Chat Completions, Responses, and Anthropic Messages routes need `completions`. If the policy is attached but the active capability has no rules, the request receives a 403 response explaining which capability to add. ## Allow-list example ```json { "name": "ai-gateway-model-filtering-inbound", "policyType": "ai-gateway-model-filtering", "handler": { "export": "AIGatewayModelFilteringInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "models": { "completions": { "allowList": ["openai/gpt-4o-mini", "anthropic/claude-haiku-4-5"] }, "embeddings": { "allowList": ["openai/text-embedding-3-small"] } } } } } ``` ## Block-list example ```json { "models": { "completions": { "blockList": ["openai/deprecated-model", "anthropic/deprecated-model"] } } } ``` In block-list mode, an unknown Provider Name is rejected even if the model is not listed. Entries whose Provider Names are absent from the live catalog are reported as warnings because they cannot match a request. ## Request behavior | Situation | Result | | --------------------------------------------------- | ------------------------------------------------------ | | Allow list, request omits `model` | The first allow-list entry is selected. | | Allow list, request names a listed model | The matching configured entry is selected. | | Allow list, request names an unlisted model | 403 response listing the allowed models. | | Block list, request omits `model` | 400 response asking for `providerName/model`. | | Block list, request names a blocked model | 403 response. | | Either mode, request uses a bare or malformed model | 400 response explaining the required format. | | Policy has no rules for the route capability | 403 response explaining which capability to configure. | | Another inbound policy already selected routing | Model Filtering leaves that selection unchanged. | Native routes also enforce the wire format. `/v1/responses` requires a selection served on the OpenAI API, and `/v1/messages` one served on the Anthropic API. A selection qualifies when its Provider Assignment is backed by that provider type, or when the provider catalog declares the selected model's dialect as that format — Bedrock Mantle serves its `openai`-dialect models on `/v1/responses` and its `anthropic`-dialect models on `/v1/messages`, and Vertex AI serves its `anthropic`-dialect Claude models on `/v1/messages`. Provider Names may be custom labels; validation never infers anything from the label or the model name, and a model the catalog does not declare is rejected. For example, this embedding request is evaluated against `models.embeddings`: ```json { "model": "openai/text-embedding-3-small", "input": "Search text" } ``` ## Adding fallbacks Declare AI Gateway Fallback Model separately and place it immediately after this policy. Its `fallback` handles retryable errors and timeouts; `quotaFallback` handles usage-limit signals independently. ## Write your own routing policy Everything Model Filtering does is built on two public primitives, so a custom inbound policy can replace it entirely. The policy's job is to choose one allowed target and store it: - `AIGatewayModels.load(context)` returns the cached provider catalog, including each model's capability, status, and per-token pricing. - `AIGatewayModelRouting.set(context, routing)` validates the routing, resolves provider credentials internally, and stores the selection that the AI Gateway handler consumes. ```typescript import { AIGatewayModelRouting, AIGatewayModels, type ZuploContext, type ZuploRequest, } from "@zuplo/runtime"; export default async function selectModel( request: ZuploRequest, context: ZuploContext, ) { const providers = await AIGatewayModels.load(context); const openAI = providers.find( (provider) => provider.providerName === "openai", ); const model = openAI?.models.find( (candidate) => candidate.capability === "completions" && candidate.status === "active", ); if (!model) { throw new Error("No active OpenAI completions model is available"); } await AIGatewayModelRouting.set(context, { completions: `openai/${model.model}`, }); return request; } ``` Attach the module as an ordinary inbound policy instead of Model Filtering: ```json { "name": "my-model-filtering-inbound", "policyType": "custom-code-inbound", "handler": { "export": "default", "module": "$import(./modules/my-model-filtering)" } } ``` `AIGatewayModelRouting.get(context)` returns the sanitized, normalized routing for the current request and never returns credentials. The AI Gateway handler consumes a custom selection even when Model Filtering is not attached. If neither kind of policy creates a selection, the handler derives one from the request's required `providerName/model`. Policy order determines precedence: 1. Routing selected before Model Filtering remains authoritative because Model Filtering leaves an existing selection unchanged. 2. Model Filtering creates routing when no earlier policy selected it. 3. A custom policy placed after Model Filtering may deliberately replace that selection. 4. If no policy selects routing, the handler derives it from the request. Prefer one policy as the primary selector so the route's intent is easy to understand. ## Model discovery Authenticated `GET /v1/models` and `GET /v1/models/{providerName/model}` use the same app URL as inference. Select the inference endpoint with the optional `x-zuplo-models-endpoint` request header: | Value | Eligible models | Response format | | ------------------ | ----------------------------------------------------------------------------------------------- | --------------- | | `chat-completions` | Active completion models supported by Chat Completions adapters, including existing translation | OpenAI | | `messages` | Active completion models supported by native Messages passthrough | Anthropic | | `responses` | Active completion models supported by the resolved Responses adapter | OpenAI | | `embeddings` | Active embedding models supported by the resolved embedding adapter | OpenAI | An explicit header wins. Otherwise, the presence of `anthropic-version` selects `messages`; other requests default to `chat-completions`. Empty, unknown, and multiple selector values return an actionable 400. The selector affects discovery only. Embedding discovery is opt-in. Messages never translates to Chat Completions. Discovery applies the first Model Filtering policy's parsed rules, matching inference's selection precedence. A capability omitted from that policy lists no models. Without Model Filtering, discovery lists the eligible catalog. Inactive models, unavailable adapters, and assignments without locally resolvable credentials are excluded. List and retrieve use identical eligibility rules; unknown, filtered-out, and wrong-endpoint IDs return the same 404. IDs retain the exact model-name casing; only the provider assignment label is normalized. OpenAI entries contain `id`, `object`, `created`, and `owned_by`. `owned_by` means the provider assignment namespace, not the model's developer. Anthropic entries contain `id`, `type`, `display_name`, and `created_at`, with unknown required nullable metadata set to `null`. Unknown creation dates use the epoch; optional unknown metadata is omitted. OpenAI lists return every eligible model and ignore pagination parameters. Anthropic lists default to 20 entries, accept `limit` from 1–1000, and accept either `after_id` or `before_id` from a previous page. Follow `has_more` and `last_id` for forward pagination. Invalid cursors return 400; restart without a cursor. Discovery uses the existing 60-second catalog cache. An uncached lookup, including legacy fallback, has a two-second budget; failure returns a sanitized 503. A successfully loaded catalog with no eligible models returns an empty 200. Responses use `Cache-Control: private, no-store` and vary on both selector headers. Only GET is supported. POST, PUT, PATCH, DELETE, and HEAD return 405 with `Allow: GET`. OPTIONS is handled by the gateway’s existing CORS preflight handler before app routing. Authentication and general abuse protections still apply. Discovery skips prompt inspection, semantic caching, and inference-budget blocking, and records one baseline request. Eligibility describes the catalog, adapters, and declared filtering rules; it does not guarantee upstream availability, provider acceptance of credentials, or acceptance by prompt-dependent custom policies. Existing Chat Completions translation does not promise full feature parity. Keep a qualified model configured when a client's picker does not consume discovery. Read more about [how policies work](/articles/policies)