# AI Gateway Fallback Model (v2) Policy

AI Gateway Fallback Model (v2) adds retry, timeout, and quota fallbacks to an
existing model selection. Place it after AI Gateway Model Filtering (v2). It
preserves the primary selection and never creates one when the policy chain is
misordered.

:::caution{title="Beta"}

This policy is in beta. You can use it today, but it may change in non-backward compatible ways before the final release.

:::

## Configuration

The configuration shows how to configure the policy in the 'policies.json' document.

```json title="config/policies.json"
{
  "name": "my-ai-gateway-fallback-model-v2-inbound-policy",
  "policyType": "ai-gateway-fallback-model-v2-inbound",
  "handler": {
    "export": "AIGatewayFallbackModelV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "fallbackTimeoutSeconds": 60,
      "models": {
        "completions": {},
        "embeddings": {}
      }
    }
  }
}
```

### Policy Configuration

- `name` <code className="text-green-600">&lt;string&gt;</code> - The name of your policy instance. This is used as a reference in your routes.
- `policyType` <code className="text-green-600">&lt;string&gt;</code> - The identifier of the policy. This is used by the Zuplo UI. Value should be `ai-gateway-fallback-model-v2-inbound`.
- `handler.export` <code className="text-green-600">&lt;string&gt;</code> - The name of the exported type. Value should be `AIGatewayFallbackModelV2InboundPolicy`.
- `handler.module` <code className="text-green-600">&lt;string&gt;</code> - The module containing the policy. Value should be `$import(@zuplo/runtime)`.
- `handler.options` <code className="text-green-600">&lt;object&gt;</code> - The options for this policy. [See Policy Options](#policy-options) below.

### Policy Options

The options for this policy are specified below. All properties are optional unless specifically marked as required.

- `models` **(required)** <code className="text-green-600">&lt;object&gt;</code> - Fallbacks grouped by AI Gateway capability.
  - `completions` <code className="text-green-600">&lt;object&gt;</code> - No description available.
    - `fallback` <code className="text-green-600">&lt;string&gt;</code> - The model attempted after a retryable error or timeout.
    - `quotaFallback` <code className="text-green-600">&lt;string&gt;</code> - The model selected after a usage-limit-exceeded signal.
  - `embeddings` <code className="text-green-600">&lt;object&gt;</code> - No description available.
    - `fallback` <code className="text-green-600">&lt;string&gt;</code> - The model attempted after a retryable error or timeout.
    - `quotaFallback` <code className="text-green-600">&lt;string&gt;</code> - The model selected after a usage-limit-exceeded signal.
- `fallbackTimeoutSeconds` <code className="text-green-600">&lt;number&gt;</code> - How long a primary model may take before the failure fallback is attempted. Defaults to `60`.

## Using the Policy

# AI Gateway Fallback Model (v2) Policy

Use this policy to add resilience to a model selection created by Model
Filtering or a custom routing policy. Place it after Model Filtering:

```text
Model Filtering -> Fallback Model -> AI Gateway handler
```

Fallback Model never creates a primary selection. If it runs without a prior
selection, it logs a warning and leaves the request unchanged.

## Options

`models` must contain `completions`, `embeddings`, or both. Each configured
capability must set at least one of:

- `fallback`: the `providerName/model` attempted after a retryable error or
  after the primary exceeds `fallbackTimeoutSeconds`.
- `quotaFallback`: the `providerName/model` selected after a
  usage-limit-exceeded signal. This path is independent of retry and timeout
  fallback.

`fallbackTimeoutSeconds` applies to every configured `fallback`. It defaults to
60 seconds and accepts values from 1 through 300.

## Cross-provider fallback example

```json
{
  "name": "ai-gateway-fallback-model-v2-inbound",
  "policyType": "ai-gateway-fallback-model-v2",
  "handler": {
    "export": "AIGatewayFallbackModelV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "models": {
        "completions": {
          "fallback": "anthropic/claude-haiku-4-5",
          "quotaFallback": "openai/gpt-4o-mini"
        },
        "embeddings": {
          "fallback": "openai/text-embedding-3-small"
        }
      },
      "fallbackTimeoutSeconds": 30
    }
  }
}
```

This configuration preserves the primary model selected earlier in the chain.
For completions, Anthropic becomes the retry/timeout backup and OpenAI becomes
the independent quota fallback. The gateway validates every reference against
the live provider catalog and resolves credentials when the selection is set.

## Composition behavior

Only capabilities already present in the current selection are modified.
Capabilities not configured here are preserved, as are fields this policy does
not override. A configured `fallback` replaces an existing backup and timeout; a
configured `quotaFallback` replaces an existing quota fallback.

Retry and timeout fallback is available for translated Chat Completions and
Embeddings requests. Native `/v1/messages` and `/v1/responses` requests are
passed through without retrying a backup. Quota fallback remains a separate,
metering-driven path.

## Write your own fallback policy

Fallback Model also uses the public routing primitives, so custom code can
choose fallbacks dynamically:

- `AIGatewayModels.load(context)` returns the cached provider catalog, including
  each model's capability, status, and per-token pricing.
- `AIGatewayModelRouting.get(context)` returns the sanitized, normalized routing
  selected earlier in the chain. It never returns provider credentials.
- `AIGatewayModelRouting.set(context, routing)` validates every target, resolves
  provider credentials internally, and replaces the complete stored selection.

Because `set()` replaces rather than partially updates routing, a custom
fallback policy must read, merge, and set. It should leave the request unchanged
when no earlier policy created a selection:

```typescript
import {
  AIGatewayModelRouting,
  AIGatewayModels,
  type ZuploContext,
  type ZuploRequest,
} from "@zuplo/runtime";

export default async function addFallback(
  request: ZuploRequest,
  context: ZuploContext,
) {
  const routing = AIGatewayModelRouting.get(context);
  const current = routing?.completions;
  if (!routing || !current) {
    return request;
  }

  const providers = await AIGatewayModels.load(context);
  const anthropic = providers.find(
    (provider) => provider.providerName === "anthropic",
  );
  const fallback = anthropic?.models.find(
    (candidate) =>
      candidate.capability === "completions" && candidate.status === "active",
  );
  if (!fallback) {
    return request;
  }

  await AIGatewayModelRouting.set(context, {
    ...routing,
    completions: {
      ...(typeof current === "string" ? { main: current } : current),
      backup: `anthropic/${fallback.model}`,
      fallbackTimeoutSeconds: 30,
    },
  });
  return request;
}
```

Place this custom policy after Model Filtering, replacing Fallback Model in the
inbound chain. Keeping the no-selection guard prevents a misordered custom
policy from creating a primary selection and bypassing filtering.

Read more about [how policies work](/articles/policies)