Copy page

Smart Router Policy

Classifies the last user prompt by calling a dedicated classifier AI Gateway app, stores the result on AIGatewaySmartRouter for later policies, and optionally routes completions by classified complexity. Classifier failures fail open so the original request still reaches the model.

Configuration

The configuration shows how to configure the policy in the 'policies.json' document.

Code
{
  "name": "my-ai-gateway-smart-router-inbound-policy",
  "policyType": "ai-gateway-smart-router-inbound",
  "handler": {
    "export": "AIGatewaySmartRouterInboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "classifierAppID": "$env(CLASSIFIER_APP_ID)",
      "classifierAppApiKey": "$env(CLASSIFIER_APP_API_KEY)",
      "classifierModel": "openai/gpt-4o-mini",
      "smartRoutingEnabled": true,
      "modelsByComplexity": {
        "low": "openai/gpt-4o-mini",
        "medium": "openai/gpt-4o",
        "high": "openai/gpt-5"
      }
    }
  }
}

Policy Configuration

• name <string> - The name of your policy instance. This is used as a reference in your routes.
• policyType <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be ai-gateway-smart-router-inbound.
• handler.export <string> - The name of the exported type. Value should be AIGatewaySmartRouterInboundPolicy.
• handler.module <string> - The module containing the policy. Value should be $import(@zuplo/runtime).
• handler.options <object> - The options for this policy. See Policy Options below.

Policy Options

The options for this policy are specified below. All properties are optional unless specifically marked as required.

• classifierAppID (required) <string> - The AI Gateway application id whose /v1/chat/completions route runs the classifier. The policy calls /{classifierAppID}/v1/chat/completions via context.invokeRoute.
• classifierAppApiKey (required) <string> - API key sent as Authorization: Bearer when invoking the classifier app.
• classifierModel (required) <string> - The providerName/model reference the classifier chat/completions request should use.
• smartRoutingEnabled <boolean> - When true, apply model routing from modelsByComplexity when confidence is high enough and the intent is known. Classification still runs when false. Defaults to false.
• modelsByComplexity <object> - Optional providerName/model routing by classified complexity. Used only when smartRoutingEnabled is true.
  • low <string> - Model used for prompts classified as low complexity.
  • medium <string> - Model used for prompts classified as medium complexity.
  • high <string> - Model used for prompts classified as high complexity.
• intents <object[]> - Labels the classifier may assign. Used to build the JSON schema enum and injected into classifierPrompt at {{intents}}. Omit to use the built-in taxonomy (code, summarization, translation, qa, conversation, classification, creative_writing, agentic, document_qa, other).
  • id (required) <string> - Stable intent identifier written into the classifier JSON schema enum.
  • description (required) <string> - Short description shown to the classifier for this intent.
• classifierPrompt <undefined> - System prompt for the classifier. If it includes {{intents}}, that placeholder is replaced with the configured intent id/description list. A string or an array of lines (joined with newlines). Omit to use the built-in classifier prompt.
• minConfidenceForRouting <number> - Minimum confidence (0–1) required before applying model routing. Unknown intents are capped strictly below this threshold. Defaults to 0.5.
• classifierTimeoutMs <integer> - How long to wait for the classifier invokeRoute call before skipping classification and forwarding the original request. Defaults to 8000.
• maxPromptChars <integer> - Maximum characters of user prompt sent to the classifier. Longer prompts are truncated. Defaults to 8000.

Using the Policy

AI Gateway Smart Router

Use this policy to classify the last user prompt on Chat Completions, Responses, and Anthropic Messages requests. It calls a dedicated AI Gateway application (/{classifierAppID}/v1/chat/completions) and stores the result on AIGatewaySmartRouter for later policies in the same request.

When smartRoutingEnabled is true, it overwrites completions routing from modelsByComplexity. Place it after Model Filtering so an invalid client model is still rejected before classification runs. The classified model then replaces that selection.

Classification is optional. Timeouts, classifier errors, unreadable bodies, and invalid options fail open: the original request is forwarded.

Loop prevention. The classifier hop is an invokeRoute sub-request. The policy no-ops when context.parentContext is set, so the classifier app can share the same route chain without classifying its own request.

Required options

• classifierAppID — AI Gateway application id whose chat/completions route runs the classifier.
• classifierAppApiKey — bearer token for that app. Use $env(CLASSIFIER_APP_API_KEY).
• classifierModel — providerName/model sent on the classifier request.

Omit intents and classifierPrompt to use the built-in taxonomy (code, summarization, translation, qa, conversation, classification, creative_writing, agentic, document_qa, other) and the built-in system prompt. Include {{intents}} in a custom prompt to inject the configured intent list.

Example

Code
{
  "name": "ai-gateway-smart-router-inbound",
  "policyType": "ai-gateway-smart-router",
  "handler": {
    "export": "AIGatewaySmartRouterInboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "classifierAppID": "$env(CLASSIFIER_APP_ID)",
      "classifierAppApiKey": "$env(CLASSIFIER_APP_API_KEY)",
      "classifierModel": "openai/gpt-4o-mini",
      "smartRoutingEnabled": true,
      "modelsByComplexity": {
        "low": "openai/gpt-4o-mini",
        "medium": "openai/gpt-4o",
        "high": "openai/gpt-5"
      }
    }
  }
}

Policy order

Code
Model Filtering -> Smart Router -> Fallback Model -> AI Gateway handler

Smart Router always set()s routing when smart routing applies, even if filtering already selected a model. Filtering skips when routing is already set, so putting this policy first would also skip allow-list checks on the client's original model.

Read the result from custom code

Code
import { AIGatewaySmartRouter } from "@zuplo/runtime";

const result = AIGatewaySmartRouter.get(context);
if (result?.profile.intent === "code") {
  // ...
}

How prompt text is chosen

The policy reads the typed body from the route (getFormat / getRequestBody), then takes the last real user text:

• /v1/chat/completions — OpenAI chat messages[], even when the downstream provider is Anthropic or Google (Zuplo translates below the policy chain).
• /v1/responses — OpenAI Responses input.
• /v1/messages — native Anthropic messages[].

Tool follow-up turns (tool / function roles, tool_result blocks, function_call_output items) are skipped so a prior user message is classified instead. Embeddings and other non-AI paths are skipped.

Fail-open behavior

The policy never 500s the user request for an internal classifier problem. Invalid options, classifier timeouts, empty classifier responses, and smart routing catalog errors are logged and the original request continues. Chat Completions, Responses, and Anthropic Messages are classified automatically. Embeddings and other non-AI paths are skipped.

Read more about how policies work