Copy page

Usage & Cost

AI Gateway Metering Policy

The AI Gateway Metering policy records each application's spend, tokens, and requests and enforces spend, token, and request budgets. When a limit is exceeded, it uses the model selection's quota fallback when one is configured or returns 429 Too Many Requests.

A budget rule covers the whole application, or gives every distinct value of an expression its own budget, such as one budget per user with request.headers.get("x-user-id").

Place it after Model Filtering and Fallback Model, and before policies such as Semantic Cache that may answer without calling a provider. Metering fails open by default when its service is unavailable; set throwOnFailure to true to reject requests instead.

Configuration

The configuration shows how to configure the policy in the 'policies.json' document.

Code
{
  "name": "my-ai-gateway-metering-v2-inbound-policy",
  "policyType": "ai-gateway-metering-v2-inbound",
  "handler": {
    "export": "AIGatewayMeteringV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "budgetRules": [
        {
          "budgetBy": "app",
          "meters": [
            {
              "meter": "cost",
              "period": "monthly",
              "value": 100,
              "action": "block"
            },
            {
              "meter": "cost",
              "period": "monthly",
              "value": 80,
              "action": "warn"
            }
          ]
        }
      ],
      "throwOnFailure": false
    }
  }
}

Policy Configuration

• name <string> - The name of your policy instance. This is used as a reference in your routes.
• policyType <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be ai-gateway-metering-v2-inbound.
• handler.export <string> - The name of the exported type. Value should be AIGatewayMeteringV2InboundPolicy.
• handler.module <string> - The module containing the policy. Value should be $import(@zuplo/runtime).
• handler.options <object> - The options for this policy. See Policy Options below.

Policy Options

The options for this policy are specified below. All properties are optional unless specifically marked as required.

• throwOnFailure <boolean> - Throw when the metering service is unavailable instead of failing open. Defaults to false.
• budgetRules <object[]> - Budget rules for this application. Each rule budgets the whole app ("app") or each distinct value of an expression ("expression"), with warn and block thresholds per meter and period.
  • budgetBy (required) <string> - Budget the application as a whole ("app"), or give every distinct value of an expression its own budget ("expression"). Allowed values are app, expression.
  • expression <string> - Required when budgetBy is "expression"; forbidden for "app". Each distinct value gets its own budget. Select one value from the request or its context: request.headers.get("x-user-id"), request.query.tenant, request.params.customerId, request.user.sub, request.user.data.organizationId, request.method, request.url, context.custom.tenantId, context.route.path, or context.incomingRequestProperties.country. Comparisons, arithmetic, string methods, and request body values are not supported.
  • meters (required) <object[]> - Thresholds this rule enforces. Add one entry per meter, period, and action you want.
    • meter (required) <string> - What to measure: US dollars spent ("cost"), requests made ("requests"), or tokens used ("tokens"). Allowed values are cost, requests, tokens.
    • period (required) <string> - How often usage resets. Allowed values are hourly, daily, weekly, monthly.
    • value (required) <number> - The threshold for this meter and period. Cost is in US dollars.
    • action (required) <string> - What happens at the threshold: "warn" notifies and lets the request through, "block" uses the configured quota fallback model or returns 429 Too Many Requests. Allowed values are warn, block.
• limits <object> - Usage limits grouped by meter.
  • costs <object> - No description available.
    • daily <object> - No description available.
      • enabled (required) <boolean> - No description available.
      • limit <number> - No description available.
      • warning <object> - No description available.
        • enabled (required) <boolean> - No description available.
        • threshold <number> - Percentage of the limit at which to emit a warning event.
    • monthly <object> - No description available.
      • enabled (required) <boolean> - No description available.
      • limit <number> - No description available.
      • warning <object> - No description available.
        • enabled (required) <boolean> - No description available.
        • threshold <number> - Percentage of the limit at which to emit a warning event.
  • tokens <object> - No description available.
    • daily <object> - No description available.
      • enabled (required) <boolean> - No description available.
      • limit <number> - No description available.
      • warning <object> - No description available.
        • enabled (required) <boolean> - No description available.
        • threshold <number> - Percentage of the limit at which to emit a warning event.
    • monthly <object> - No description available.
      • enabled (required) <boolean> - No description available.
      • limit <number> - No description available.
      • warning <object> - No description available.
        • enabled (required) <boolean> - No description available.
        • threshold <number> - Percentage of the limit at which to emit a warning event.
  • requests <object> - No description available.
    • daily <object> - No description available.
      • enabled (required) <boolean> - No description available.
      • limit <number> - No description available.
      • warning <object> - No description available.
        • enabled (required) <boolean> - No description available.
        • threshold <number> - Percentage of the limit at which to emit a warning event.
    • monthly <object> - No description available.
      • enabled (required) <boolean> - No description available.
      • limit <number> - No description available.
      • warning <object> - No description available.
        • enabled (required) <boolean> - No description available.
        • threshold <number> - Percentage of the limit at which to emit a warning event.

Using the Policy

AI Gateway Metering records application usage and configures the app's own budgets before the provider request runs. It meters spend, tokens, and requests. Budget rules can cover the whole application or each distinct value of an expression.

Application, team, and gateway budgets are enforced even when this policy is not included in the application's policy chain.

Request logs indicate when automatic budget checks run and when a budget blocks a request or selects a fallback model.

When a budget blocks a provider request, the gateway uses the configured quota fallback model or returns 429 Too Many Requests. Cached responses are also subject to budgets: a blocked cache hit returns 429 Too Many Requests without using a fallback model.

Place Metering after Model Filtering and Fallback Model. Place it before Semantic Cache so cache hits count toward request limits.

Example

Code
{
  "name": "ai-gateway-metering-v2-inbound",
  "options": {
    "throwOnFailure": false,
    "budgetRules": [
      {
        "budgetBy": "app",
        "meters": [
          {
            "meter": "cost",
            "period": "monthly",
            "value": 100,
            "action": "block"
          },
          {
            "meter": "cost",
            "period": "monthly",
            "value": 80,
            "action": "warn"
          }
        ]
      },
      {
        "budgetBy": "expression",
        "expression": "request.headers.get(\"x-user\")",
        "meters": [
          {
            "meter": "cost",
            "period": "daily",
            "value": 5,
            "action": "block"
          }
        ]
      }
    ]
  }
}

The supported meters are cost, requests, and tokens. The supported periods are hourly, daily, weekly, and monthly. An application can have at most five rules. An expression rule creates a separate budget for every distinct value of its expression. An action of "warn" notifies without blocking. An action of "block" activates the configured quota fallback or returns 429 Too Many Requests when usage reaches the value.

Budget expressions

A rule with "budgetBy": "expression" gives every distinct value of its expression its own budget. A daily cost budget of five dollars on request.headers.get("x-user-id") gives each user five dollars a day, rather than five dollars shared across all users.

An expression selects one value from the request or its context. It is a selector, not code: there are no comparisons, arithmetic, string concatenation, or method calls other than the header accessor below.

Supported expressions

Expression	Example	Selects
request.headers.get("<name>")	request.headers.get("x-user-id")	One request header. Header names are case-insensitive.
request.query.<name>	request.query.tenant	One query-string parameter.
request.searchParams.<name>	request.searchParams.tenant	One query-string parameter.
request.params.<name>	request.params.customerId	One path parameter from the matched route.
request.user.sub	request.user.sub	The authenticated consumer's subject, such as the API key name.
request.user.data.<property>	request.user.data.organizationId	A property of the consumer's metadata, including nested ones.
request.<property>	request.method	A scalar request property.
context.custom.<property>	context.custom.tenantId	A value your own policies or handlers put on context.custom.
context.route.<property>	context.route.path	A property of the matched route, such as path or label.
context.incomingRequestProperties.<name>	context.incomingRequestProperties.country	A connection property, such as country, city, or asn.

The selectable scalar request properties are bodyUsed, cache, credentials, destination, integrity, keepalive, method, mode, redirect, referrer, referrerPolicy, and url.

Use quoted brackets for any key that is not a plain identifier: request.query["team-id"], context.custom["tenant.id"].

context.incomingRequestProperties exposes connection data only: country, city, region, regionCode, continent, colo, latitude, longitude, postalCode, metroCode, timezone, ip, asn, asOrganization, and httpProtocol. Client-certificate and mTLS verification fields are not selectable. To budget per client certificate, run the mTLS Authentication policy first and select the metadata it attaches, such as request.user.data.mtlsAuth.sha256Fingerprint.

An expression is limited to 1024 bytes and 32 property segments.

Examples

Give every end user their own daily spend budget, keyed by a header your application sends:

Code
{
  "budgetBy": "expression",
  "expression": "request.headers.get(\"x-user-id\")",
  "meters": [
    { "meter": "cost", "period": "daily", "value": 5, "action": "block" }
  ]
}

Give every customer of yours their own monthly token budget, keyed by metadata on the API key that made the request:

Code
{
  "budgetBy": "expression",
  "expression": "request.user.data.organizationId",
  "meters": [
    {
      "meter": "tokens",
      "period": "monthly",
      "value": 8000000,
      "action": "warn"
    },
    {
      "meter": "tokens",
      "period": "monthly",
      "value": 10000000,
      "action": "block"
    }
  ]
}

Cap requests per tenant per hour, keyed by a path parameter on a route such as /tenants/:tenantId/chat:

Code
{
  "budgetBy": "expression",
  "expression": "request.params.tenantId",
  "meters": [
    {
      "meter": "requests",
      "period": "hourly",
      "value": 1000,
      "action": "block"
    }
  ]
}

Values

An expression must resolve to a string or a safe integer. The runtime converts the value to well-formed NFC Unicode, and ignores it when it exceeds 256 UTF-8 bytes or contains control characters, U+2028, or U+2029. Values are case-sensitive, so Acme and acme get separate budgets, but two strings that normalize to the same NFC text share one budget.

When an expression resolves to nothing, or to something else such as an object, an array, or a fractional number, that rule does not count or block the request. Other rules still apply. Pair a rule on a value you must enforce with a policy that rejects requests missing it, such as API key authentication or request validation.

Not supported

Not supported	Use instead
Request body values, such as request.body.userId	A header, a query parameter, a path parameter, or API key metadata.
Whole objects, such as request.headers	One value, such as request.headers.get("x-user-id").
Comparisons, &&, ternaries, and literals	One selector per rule. An application can have up to five rules.
String methods, such as .toLowerCase()	The value as the client sends it.
Wildcards and multiple keys in one expression	One rule per value you want to budget.
Arrays, class instances, and inherited properties	A plain value the gateway or your own policy sets.
Per-request identifiers, such as context.requestId	A value shared by many requests, such as a user or tenant.

An unsupported expression does not fail the request. The rule is skipped, and the request is not budgeted by it.

Keep the spelling stable

Each expression is stored exactly as you write it, and its budgets are tracked under that exact text. request.headers.get("x-user-id") and request.headers["x-user-id"] read the same header but keep separate budgets, and editing a rule's expression starts its budgets over. Single quotes and double quotes likewise identify different rules. Use the spellings in the table above.

Several forms parse, so prefer the canonical one. A header is written request.headers.get("content-type"), with a lowercase name; the equivalent request.headers.content-type and request.headers["content-type"] are accepted but keep separate budgets. A dot property starts with a letter, _, or $; a key containing anything else takes quoted brackets, so context.custom.team-id is canonically context.custom["team-id"].

In raw JSON, escape the double quotes a bracket segment needs:

Code
{
  "expression": "request.user.data[\"team-id\"]"
}

After a JSON parser decodes that value the expression is request.user.data["team-id"]; the backslashes are not part of its identity.

Team limits

Application budgets apply only to that application. Parent team and gateway budgets apply across their applications, even when an application does not include Metering. An application cannot override a shared budget with a higher limit of its own.

For example, two applications that spend $40 and $10 exhaust their team's $50 monthly budget. An application outside that team does not share its budget.

Team policy templates provide defaults for application policies. To set a shared team budget, configure the budget on the team itself.

Budget availability

Budget changes can take time to take effect. Usage is recorded asynchronously, so concurrent requests can exceed a budget before further requests are blocked.

If a budget check is unavailable, requests can proceed unless a budget has already been determined to block them. The throwOnFailure option controls whether this policy rejects requests when its metering operations fail; it does not change how application, team, or gateway budgetRules handle unavailable checks.

Read more about how policies work