Cookbook: Custom fallback logic

The Fallback Model policy covers a fixed backup and quota fallback. When the backup should be chosen dynamically—the cheapest available model, a provider-specific preference, whatever is currently active—a custom chain policy can replace it using the same public primitives.

A fallback policy enriches the selection an earlier policy created. Because AIGatewayModelRouting.set() replaces the complete stored selection rather than partially updating it, the policy must read, merge, and set. It should also leave the request unchanged when no earlier policy created a selection—so a misplaced chain entry doesn't bypass model filtering—and when the chosen backup equals the current main model, which set() rejects.

The policy

This example adds the first active Anthropic completions model as the backup, with a 30-second timeout:

modules/add-fallback.ts
import {
  AIGatewayModelRouting,
  AIGatewayModels,
  ZuploContext,
  ZuploRequest,
} from "@zuplo/runtime";

export default async function addFallback(
  request: ZuploRequest,
  context: ZuploContext,
): Promise<ZuploRequest> {
  const routing = AIGatewayModelRouting.get(context);
  const current = routing?.completions;
  if (!routing || !current) {
    return request;
  }
  const target = typeof current === "string" ? { main: current } : current;

  const providers = await AIGatewayModels.load(context);
  const anthropic = providers.find(
    (provider) => provider.providerName === "anthropic",
  );
  const fallback = anthropic?.models.find(
    (candidate) =>
      candidate.capability === "completions" && candidate.status === "active",
  );
  const backup = fallback ? `anthropic/${fallback.model}` : undefined;
  if (!backup || backup.toLowerCase() === target.main.toLowerCase()) {
    return request;
  }

  await AIGatewayModelRouting.set(context, {
    ...routing,
    completions: {
      ...target,
      backup,
      fallbackTimeoutSeconds: 30,
    },
  });
  return request;
}

Declare and use it

config/policies.json (one entry in the policies array)
{
  "name": "add-fallback",
  "policyType": "custom-code-inbound",
  "handler": {
    "export": "default",
    "module": "$import(./modules/add-fallback)"
  }
}

Add add-fallback to an app's chain directly after Model Filtering, replacing the Fallback Model policy. Keep the no-selection guard at the top of the policy—it prevents a misplaced entry from creating a primary selection and bypassing filtering.

Next steps

• Fallback Models: the built-in fallback and quota-fallback behavior
• Cookbook: Dynamic model routing: create the primary selection from code
• Custom Policies: the full custom-policy quickstart