# AI Gateway Metering Policy

:::note{title="AI Gateway Policy"}

This policy is for use with the [AI Gateway](/docs/ai-gateway/overview). See
the AI Gateway documentation to learn how to configure and govern AI models
with Zuplo.

:::

The AI Gateway Metering policy records each application's spend, tokens, and
requests and enforces spend, token, and request budgets. When a limit is
exceeded, it uses the model selection's quota fallback when one is configured or
returns `429 Too Many Requests`.

A budget rule covers the whole application, or gives every distinct value of an
expression its own budget, such as one budget per user with
`request.headers.get("x-user-id")`.

Place it after Model Filtering and Fallback Model, and before policies such as
Semantic Cache that may answer without calling a provider. Metering fails open
by default when its service is unavailable; set `throwOnFailure` to `true` to
reject requests instead.

## Configuration

The configuration shows how to configure the policy in the 'policies.json' document.

```json title="config/policies.json"
{
  "name": "my-ai-gateway-metering-inbound-policy",
  "policyType": "ai-gateway-metering-inbound",
  "handler": {
    "export": "AIGatewayMeteringInboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "budgetRules": [
        {
          "budgetBy": "app",
          "meters": [
            {
              "meter": "cost",
              "period": "monthly",
              "value": 100,
              "action": "block"
            },
            {
              "meter": "cost",
              "period": "monthly",
              "value": 80,
              "action": "warn"
            }
          ]
        }
      ],
      "throwOnFailure": false
    }
  }
}
```

### Policy Configuration

- `name` <code className="text-green-600">&lt;string&gt;</code> - The name of your policy instance. This is used as a reference in your routes.
- `policyType` <code className="text-green-600">&lt;string&gt;</code> - The identifier of the policy. This is used by the Zuplo UI. Value should be `ai-gateway-metering-inbound`.
- `handler.export` <code className="text-green-600">&lt;string&gt;</code> - The name of the exported type. Value should be `AIGatewayMeteringInboundPolicy`.
- `handler.module` <code className="text-green-600">&lt;string&gt;</code> - The module containing the policy. Value should be `$import(@zuplo/runtime)`.
- `handler.options` <code className="text-green-600">&lt;object&gt;</code> - The options for this policy. [See Policy Options](#policy-options) below.

### Policy Options

The options for this policy are specified below. All properties are optional unless specifically marked as required.

- `throwOnFailure` <code className="text-green-600">&lt;boolean&gt;</code> - Throw when the metering service is unavailable instead of failing open. Defaults to `false`.
- `budgetRules` <code className="text-green-600">&lt;object[]&gt;</code> - Budget rules for this application. Each rule budgets the whole app ("app") or each distinct value of an expression ("expression"), with warn and block thresholds per meter and period.
  - `budgetBy` **(required)** <code className="text-green-600">&lt;string&gt;</code> - Budget the application as a whole ("app"), or give every distinct value of an expression its own budget ("expression"). Allowed values are `app`, `expression`.
  - `expression` <code className="text-green-600">&lt;string&gt;</code> - Required when budgetBy is "expression"; forbidden for "app". Each distinct value gets its own budget. Select one value from the request or its context: request.headers.get("x-user-id"), request.query.tenant, request.params.customerId, request.user.sub, request.user.data.organizationId, request.method, request.url, context.custom.tenantId, context.route.path, or context.incomingRequestProperties.country. Comparisons, arithmetic, string methods, and request body values are not supported.
  - `meters` **(required)** <code className="text-green-600">&lt;object[]&gt;</code> - Thresholds this rule enforces. Add one entry per meter, period, and action you want. An "app" rule needs at least one entry. An "expression" rule may leave this empty to track usage per value without enforcing a limit.
    - `meter` **(required)** <code className="text-green-600">&lt;string&gt;</code> - What to measure: US dollars spent ("cost"), requests made ("requests"), or tokens used ("tokens"). Allowed values are `cost`, `requests`, `tokens`.
    - `period` **(required)** <code className="text-green-600">&lt;string&gt;</code> - How often usage resets. Allowed values are `hourly`, `daily`, `weekly`, `monthly`.
    - `value` **(required)** <code className="text-green-600">&lt;number&gt;</code> - The threshold for this meter and period. Cost is in US dollars.
    - `action` **(required)** <code className="text-green-600">&lt;string&gt;</code> - What happens at the threshold: "warn" notifies and lets the request through, "block" uses the configured quota fallback model or returns 429 Too Many Requests. Allowed values are `warn`, `block`.
- `limits` <code className="text-green-600">&lt;object&gt;</code> - Usage limits grouped by meter.
  - `costs` <code className="text-green-600">&lt;object&gt;</code> - No description available.
    - `daily` <code className="text-green-600">&lt;object&gt;</code> - No description available.
      - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
      - `limit` <code className="text-green-600">&lt;number&gt;</code> - No description available.
      - `warning` <code className="text-green-600">&lt;object&gt;</code> - No description available.
        - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
        - `threshold` <code className="text-green-600">&lt;number&gt;</code> - Percentage of the limit at which to emit a warning event.
    - `monthly` <code className="text-green-600">&lt;object&gt;</code> - No description available.
      - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
      - `limit` <code className="text-green-600">&lt;number&gt;</code> - No description available.
      - `warning` <code className="text-green-600">&lt;object&gt;</code> - No description available.
        - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
        - `threshold` <code className="text-green-600">&lt;number&gt;</code> - Percentage of the limit at which to emit a warning event.
  - `tokens` <code className="text-green-600">&lt;object&gt;</code> - No description available.
    - `daily` <code className="text-green-600">&lt;object&gt;</code> - No description available.
      - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
      - `limit` <code className="text-green-600">&lt;number&gt;</code> - No description available.
      - `warning` <code className="text-green-600">&lt;object&gt;</code> - No description available.
        - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
        - `threshold` <code className="text-green-600">&lt;number&gt;</code> - Percentage of the limit at which to emit a warning event.
    - `monthly` <code className="text-green-600">&lt;object&gt;</code> - No description available.
      - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
      - `limit` <code className="text-green-600">&lt;number&gt;</code> - No description available.
      - `warning` <code className="text-green-600">&lt;object&gt;</code> - No description available.
        - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
        - `threshold` <code className="text-green-600">&lt;number&gt;</code> - Percentage of the limit at which to emit a warning event.
  - `requests` <code className="text-green-600">&lt;object&gt;</code> - No description available.
    - `daily` <code className="text-green-600">&lt;object&gt;</code> - No description available.
      - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
      - `limit` <code className="text-green-600">&lt;number&gt;</code> - No description available.
      - `warning` <code className="text-green-600">&lt;object&gt;</code> - No description available.
        - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
        - `threshold` <code className="text-green-600">&lt;number&gt;</code> - Percentage of the limit at which to emit a warning event.
    - `monthly` <code className="text-green-600">&lt;object&gt;</code> - No description available.
      - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
      - `limit` <code className="text-green-600">&lt;number&gt;</code> - No description available.
      - `warning` <code className="text-green-600">&lt;object&gt;</code> - No description available.
        - `enabled` **(required)** <code className="text-green-600">&lt;boolean&gt;</code> - No description available.
        - `threshold` <code className="text-green-600">&lt;number&gt;</code> - Percentage of the limit at which to emit a warning event.

## Using the Policy

AI Gateway Metering records application usage and configures the app's own
budgets before the provider request runs. It meters spend, tokens, and requests.
Budget rules can cover the whole application or each distinct value of an
expression.

Application, team, and gateway budgets are enforced even when this policy is not
included in the application's policy chain.

Request logs indicate when automatic budget checks run and when a budget blocks
a request or selects a fallback model.

When a budget blocks a provider request, the gateway uses the configured quota
fallback model or returns `429 Too Many Requests`. Cached responses are also
subject to budgets: a blocked cache hit returns `429 Too Many Requests` without
using a fallback model.

Place Metering after Model Filtering and Fallback Model. Place it before
Semantic Cache so cache hits count toward request limits.

## Example

```json
{
  "name": "ai-gateway-metering-inbound",
  "options": {
    "throwOnFailure": false,
    "budgetRules": [
      {
        "budgetBy": "app",
        "meters": [
          {
            "meter": "cost",
            "period": "monthly",
            "value": 100,
            "action": "block"
          },
          {
            "meter": "cost",
            "period": "monthly",
            "value": 80,
            "action": "warn"
          }
        ]
      },
      {
        "budgetBy": "expression",
        "expression": "request.headers.get(\"x-user\")",
        "meters": [
          {
            "meter": "cost",
            "period": "daily",
            "value": 5,
            "action": "block"
          }
        ]
      }
    ]
  }
}
```

The supported meters are `cost`, `requests`, and `tokens`. The supported periods
are `hourly`, `daily`, `weekly`, and `monthly`. An application can have at most
five rules. An expression rule creates a separate budget for every distinct
value of its expression. An action of `"warn"` notifies without blocking. An
action of `"block"` activates the configured quota fallback or returns
`429 Too Many Requests` when usage reaches the value.

## Budget expressions

A rule with `"budgetBy": "expression"` gives every distinct value of its
expression its own budget. A daily cost budget of five dollars on
`request.headers.get("x-user-id")` gives each user five dollars a day, rather
than five dollars shared across all users.

An expression selects one value from the request or its context. It is a
selector, not code: there are no comparisons, arithmetic, string concatenation,
or method calls other than the header accessor below.

### Supported expressions

| Expression                                 | Example                                     | Selects                                                         |
| ------------------------------------------ | ------------------------------------------- | --------------------------------------------------------------- |
| `request.headers.get("<name>")`            | `request.headers.get("x-user-id")`          | One request header. Header names are case-insensitive.          |
| `request.query.<name>`                     | `request.query.tenant`                      | One query-string parameter.                                     |
| `request.searchParams.<name>`              | `request.searchParams.tenant`               | One query-string parameter.                                     |
| `request.params.<name>`                    | `request.params.customerId`                 | One path parameter from the matched route.                      |
| `request.user.sub`                         | `request.user.sub`                          | The authenticated consumer's subject, such as the API key name. |
| `request.user.data.<property>`             | `request.user.data.organizationId`          | A property of the consumer's metadata, including nested ones.   |
| `request.<property>`                       | `request.method`                            | A scalar request property.                                      |
| `context.custom.<property>`                | `context.custom.tenantId`                   | A value your own policies or handlers put on `context.custom`.  |
| `context.route.<property>`                 | `context.route.path`                        | A property of the matched route, such as `path` or `label`.     |
| `context.incomingRequestProperties.<name>` | `context.incomingRequestProperties.country` | A connection property, such as `country`, `city`, or `asn`.     |

The selectable scalar request properties are `bodyUsed`, `cache`, `credentials`,
`destination`, `integrity`, `keepalive`, `method`, `mode`, `redirect`,
`referrer`, `referrerPolicy`, and `url`.

Use quoted brackets for any key that is not a plain identifier:
`request.query["team-id"]`, `context.custom["tenant.id"]`.

`context.incomingRequestProperties` exposes connection data only: `country`,
`city`, `region`, `regionCode`, `continent`, `colo`, `latitude`, `longitude`,
`postalCode`, `metroCode`, `timezone`, `ip`, `asn`, `asOrganization`, and
`httpProtocol`. Client-certificate and mTLS verification fields are not
selectable. To budget per client certificate, run the mTLS Authentication policy
first and select the metadata it attaches, such as
`request.user.data.mtlsAuth.sha256Fingerprint`.

An expression is limited to 1024 bytes and 32 property segments.

### Examples

Give every end user their own daily spend budget, keyed by a header your
application sends:

```json
{
  "budgetBy": "expression",
  "expression": "request.headers.get(\"x-user-id\")",
  "meters": [
    { "meter": "cost", "period": "daily", "value": 5, "action": "block" }
  ]
}
```

Give every customer of yours their own monthly token budget, keyed by metadata
on the API key that made the request:

```json
{
  "budgetBy": "expression",
  "expression": "request.user.data.organizationId",
  "meters": [
    {
      "meter": "tokens",
      "period": "monthly",
      "value": 8000000,
      "action": "warn"
    },
    {
      "meter": "tokens",
      "period": "monthly",
      "value": 10000000,
      "action": "block"
    }
  ]
}
```

Cap requests per tenant per hour, keyed by a path parameter on a route such as
`/tenants/:tenantId/chat`:

```json
{
  "budgetBy": "expression",
  "expression": "request.params.tenantId",
  "meters": [
    {
      "meter": "requests",
      "period": "hourly",
      "value": 1000,
      "action": "block"
    }
  ]
}
```

### Values

An expression must resolve to a string or a safe integer. The runtime converts
the value to well-formed NFC Unicode, and ignores it when it exceeds 256 UTF-8
bytes or contains control characters, U+2028, or U+2029. Values are
case-sensitive, so `Acme` and `acme` get separate budgets, but two strings that
normalize to the same NFC text share one budget.

When an expression resolves to nothing, or to something else such as an object,
an array, or a fractional number, that rule does not count or block the request.
Other rules still apply. Pair a rule on a value you must enforce with a policy
that rejects requests missing it, such as API key authentication or request
validation.

### Not supported

| Not supported                                        | Use instead                                                         |
| ---------------------------------------------------- | ------------------------------------------------------------------- |
| Request body values, such as `request.body.userId`   | A header, a query parameter, a path parameter, or API key metadata. |
| Whole objects, such as `request.headers`             | One value, such as `request.headers.get("x-user-id")`.              |
| Comparisons, `&&`, ternaries, and literals           | One selector per rule. An application can have up to five rules.    |
| String methods, such as `.toLowerCase()`             | The value as the client sends it.                                   |
| Wildcards and multiple keys in one expression        | One rule per value you want to budget.                              |
| Arrays, class instances, and inherited properties    | A plain value the gateway or your own policy sets.                  |
| Per-request identifiers, such as `context.requestId` | A value shared by many requests, such as a user or tenant.          |

An unsupported expression does not fail the request. The rule is skipped, and
the request is not budgeted by it.

### Keep the spelling stable

Each expression is stored exactly as you write it, and its budgets are tracked
under that exact text. `request.headers.get("x-user-id")` and
`request.headers["x-user-id"]` read the same header but keep separate budgets,
and editing a rule's expression starts its budgets over. Single quotes and
double quotes likewise identify different rules. Use the spellings in the table
above.

Several forms parse, so prefer the canonical one. A header is written
`request.headers.get("content-type")`, with a lowercase name; the equivalent
`request.headers.content-type` and `request.headers["content-type"]` are
accepted but keep separate budgets. A dot property starts with a letter, `_`, or
`$`; a key containing anything else takes quoted brackets, so
`context.custom.team-id` is canonically `context.custom["team-id"]`.

In raw JSON, escape the double quotes a bracket segment needs:

```json
{
  "expression": "request.user.data[\"team-id\"]"
}
```

After a JSON parser decodes that value the expression is
`request.user.data["team-id"]`; the backslashes are not part of its identity.

## Team limits

Application budgets apply only to that application. Parent team and gateway
budgets apply across their applications, even when an application does not
include Metering. An application cannot override a shared budget with a higher
limit of its own.

For example, two applications that spend $40 and $10 exhaust their team's $50
monthly budget. An application outside that team does not share its budget.

Team policy templates provide defaults for application policies. To set a shared
team budget, configure the budget on the team itself.

## Budget availability

Budget changes can take time to take effect. Usage is recorded asynchronously,
so concurrent requests can exceed a budget before further requests are blocked.

If a budget check is unavailable, requests can proceed unless a budget has
already been determined to block them. The `throwOnFailure` option controls
whether this policy rejects requests when its metering operations fail; it does
not change how application, team, or gateway `budgetRules` handle unavailable
checks.

Read more about [how policies work](/articles/policies)