# Smart Router Policy :::note{title="AI Gateway Policy"} This policy is for use with the [AI Gateway](/docs/ai-gateway/introduction). See the AI Gateway documentation to learn how to configure and govern AI models with Zuplo. ::: Classifies the last user prompt by calling a dedicated classifier AI Gateway app, stores the result on `AIGatewaySmartRouter` for later policies, and optionally routes completions by classified complexity. Classifier failures fail open so the original request still reaches the model. ## Configuration The configuration shows how to configure the policy in the 'policies.json' document. ```json title="config/policies.json" { "name": "my-ai-gateway-smart-router-inbound-policy", "policyType": "ai-gateway-smart-router-inbound", "handler": { "export": "AIGatewaySmartRouterInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "classifierAppID": "$env(CLASSIFIER_APP_ID)", "classifierAppApiKey": "$env(CLASSIFIER_APP_API_KEY)", "classifierModel": "openai/gpt-4o-mini", "smartRoutingEnabled": true, "modelsByComplexity": { "low": "openai/gpt-4o-mini", "medium": "openai/gpt-4o", "high": "openai/gpt-5" } } } } ``` ### Policy Configuration - `name` <string> - The name of your policy instance. This is used as a reference in your routes. - `policyType` <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be `ai-gateway-smart-router-inbound`. - `handler.export` <string> - The name of the exported type. Value should be `AIGatewaySmartRouterInboundPolicy`. - `handler.module` <string> - The module containing the policy. Value should be `$import(@zuplo/runtime)`. - `handler.options` <object> - The options for this policy. [See Policy Options](#policy-options) below. ### Policy Options The options for this policy are specified below. All properties are optional unless specifically marked as required. - `classifierAppID` **(required)** <string> - The AI Gateway application id whose `/v1/chat/completions` route runs the classifier. The policy calls `/{classifierAppID}/v1/chat/completions` via `context.invokeRoute`. - `classifierAppApiKey` **(required)** <string> - API key sent as `Authorization: Bearer` when invoking the classifier app. - `classifierModel` **(required)** <string> - The `providerName/model` reference the classifier chat/completions request should use. - `smartRoutingEnabled` <boolean> - When true, apply model routing from `modelsByComplexity` when confidence is high enough and the intent is known. Classification still runs when false. Defaults to `false`. - `modelsByComplexity` <object> - Optional `providerName/model` routing by classified complexity. Used only when `smartRoutingEnabled` is true. - `low` <string> - Model used for prompts classified as low complexity. - `medium` <string> - Model used for prompts classified as medium complexity. - `high` <string> - Model used for prompts classified as high complexity. - `intents` <object[]> - Labels the classifier may assign. Used to build the JSON schema enum and injected into `classifierPrompt` at `{{intents}}`. Omit to use the built-in taxonomy (code, summarization, translation, qa, conversation, classification, creative_writing, agentic, document_qa, other). - `id` **(required)** <string> - Stable intent identifier written into the classifier JSON schema enum. - `description` **(required)** <string> - Short description shown to the classifier for this intent. - `classifierPrompt` <undefined> - System prompt for the classifier. If it includes `{{intents}}`, that placeholder is replaced with the configured intent id/description list. A string or an array of lines (joined with newlines). Omit to use the built-in classifier prompt. - `minConfidenceForRouting` <number> - Minimum confidence (0–1) required before applying model routing. Unknown intents are capped strictly below this threshold. Defaults to `0.5`. - `classifierTimeoutMs` <integer> - How long to wait for the classifier `invokeRoute` call before skipping classification and forwarding the original request. Defaults to `8000`. - `maxPromptChars` <integer> - Maximum characters of user prompt sent to the classifier. Longer prompts are truncated. Defaults to `8000`. ## Using the Policy # AI Gateway Smart Router Use this policy to classify the last user prompt on Chat Completions, Responses, and Anthropic Messages requests. It calls a dedicated AI Gateway application (`/{classifierAppID}/v1/chat/completions`) and stores the result on `AIGatewaySmartRouter` for later policies in the same request. When `smartRoutingEnabled` is true, it overwrites completions routing from `modelsByComplexity`. Place it **after** Model Filtering so an invalid client model is still rejected before classification runs. The classified model then replaces that selection. Classification is optional. Timeouts, classifier errors, unreadable bodies, and invalid options fail open: the original request is forwarded. > **Loop prevention.** The classifier hop is an `invokeRoute` sub-request. The > policy no-ops when `context.parentContext` is set, so the classifier app can > share the same route chain without classifying its own request. ## Required options - `classifierAppID` — AI Gateway application id whose chat/completions route runs the classifier. - `classifierAppApiKey` — bearer token for that app. Use `$env(CLASSIFIER_APP_API_KEY)`. - `classifierModel` — `providerName/model` sent on the classifier request. Omit `intents` and `classifierPrompt` to use the built-in taxonomy (code, summarization, translation, qa, conversation, classification, creative_writing, agentic, document_qa, other) and the built-in system prompt. Include `{{intents}}` in a custom prompt to inject the configured intent list. ## Example ```json { "name": "ai-gateway-smart-router-inbound", "policyType": "ai-gateway-smart-router", "handler": { "export": "AIGatewaySmartRouterInboundPolicy", "module": "$import(@zuplo/runtime)", "options": { "classifierAppID": "$env(CLASSIFIER_APP_ID)", "classifierAppApiKey": "$env(CLASSIFIER_APP_API_KEY)", "classifierModel": "openai/gpt-4o-mini", "smartRoutingEnabled": true, "modelsByComplexity": { "low": "openai/gpt-4o-mini", "medium": "openai/gpt-4o", "high": "openai/gpt-5" } } } } ``` ## Policy order ```text Model Filtering -> Smart Router -> Fallback Model -> AI Gateway handler ``` Smart Router always `set()`s routing when smart routing applies, even if filtering already selected a model. Filtering skips when routing is already set, so putting this policy first would also skip allow-list checks on the client's original model. ## Read the result from custom code ```typescript import { AIGatewaySmartRouter } from "@zuplo/runtime"; const result = AIGatewaySmartRouter.get(context); if (result?.profile.intent === "code") { // ... } ``` ## How prompt text is chosen The policy reads the typed body from the route (`getFormat` / `getRequestBody`), then takes the last real user text: - `/v1/chat/completions` — OpenAI chat `messages[]`, even when the downstream provider is Anthropic or Google (Zuplo translates below the policy chain). - `/v1/responses` — OpenAI Responses `input`. - `/v1/messages` — native Anthropic `messages[]`. Tool follow-up turns (`tool` / `function` roles, `tool_result` blocks, `function_call_output` items) are skipped so a prior user message is classified instead. Embeddings and other non-AI paths are skipped. ## Fail-open behavior The policy never 500s the user request for an internal classifier problem. Invalid options, classifier timeouts, empty classifier responses, and smart routing catalog errors are logged and the original request continues. Chat Completions, Responses, and Anthropic Messages are classified automatically. Embeddings and other non-AI paths are skipped. Read more about [how policies work](/articles/policies)