# Isolate semantic cache entries by tenant

> **Scope of this page**
>
> This page documents the Zuplo **gateway** runtime (API, AI, and MCP gateways), not the Developer Portal.
>
> It applies to every Zuplo hosting option: Managed Edge, Managed Dedicated, and Self-Hosted. See [Hosting options](https://zuplo.com/docs/articles/hosting-options).
>
> It requires the Enterprise add-on.

The [Semantic Cache policy](../policies/semantic-cache-inbound.mdx)
(`semantic-cache-inbound`) answers a request from an earlier response when the
new request means the same thing, even when the wording differs. That makes it a
good fit in front of an LLM or a natural-language search API. When several
tenants share a route, give each tenant its own **namespace** so that one
tenant's cached answer never reaches another tenant.

A namespace partitions the cache. A lookup only matches entries stored in the
same namespace, and the policy stores a response in the namespace of the request
that produced it.

## Set the namespace for each request

Set `cacheBy` to `function` and point `cacheByFunction` at a function in your
project. The policy calls the function on each request. The function returns the
cache key and, optionally, settings for that request only:

| Property               | Required | What it does                                                                |
| ---------------------- | -------- | --------------------------------------------------------------------------- |
| `cacheKey`             | Yes      | The text the policy compares by meaning, such as the user's question.       |
| `namespace`            | No       | The partition to look up and store this request in.                         |
| `semanticTolerance`    | No       | How far apart two keys can be and still match. Smaller values match closer. |
| `expirationSecondsTtl` | No       | How many seconds to keep a response that this request stores.               |

A value the function returns overrides the policy option of the same name for
that request. A value it leaves out falls back to the policy option. With no
`namespace` from either, the policy uses the `default` namespace.

This function uses the question in the request body as the key and the
authenticated caller as the namespace:

```ts title="modules/semantic-cache-key.ts"
import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export async function tenantCacheKey(
  request: ZuploRequest,
  context: ZuploContext,
  policyName: string,
) {
  // Clone so the body is still available for the backend
  const body = await request.clone().json();

  return {
    cacheKey: body.question,
    namespace: `tenant:${request.user?.sub ?? "anonymous"}`,
  };
}
```

Read the body from `request.clone()`, as in the example. Reading the original
request consumes the body before it reaches your backend.

If the function throws, or returns no string `cacheKey`, the policy logs an
error and the request continues to the backend without caching.

## Configure the policy

Reference the function in the policy options:

```json title="config/policies.json"
{
  "name": "semantic-cache",
  "policyType": "semantic-cache-inbound",
  "handler": {
    "export": "SemanticCacheInboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "cacheBy": "function",
      "cacheByFunction": {
        "module": "$import(./modules/semantic-cache-key)",
        "export": "tenantCacheKey"
      },
      "expirationSecondsTtl": 3600,
      "returnCacheStatusHeader": true
    }
  }
}
```

Add the policy to the route after your authentication policy, so that the
function can read `request.user`:

```json title="config/routes.oas.json"
"policies": {
  "inbound": ["api-key-inbound", "semantic-cache"]
}
```

With `returnCacheStatusHeader` set to `true`, every response carries a
`zp-semantic-cache` header with `HIT` or `MISS`. Set `cacheStatusHeaderName` to
use a different header name.

## Check the results

Send a question twice with one API key, then once with a key for a different
consumer:

| Request                           | `zp-semantic-cache` |
| --------------------------------- | ------------------- |
| Consumer A asks a question        | `MISS`              |
| Consumer A asks it again          | `HIT`               |
| Consumer B asks the same question | `MISS`              |

To watch hit rates over time, open
[**Observability → Analytics**](https://portal.zuplo.com/+/account/project/analytics)
in your project and select the [Cache](../analytics/tabs/cache.md) section. The
**Policies** table shows hits, misses, and hit rate for each caching policy.

## AI Gateway apps

For traffic through the AI Gateway, use the
[AI Gateway Semantic Cache](../policies/ai-gateway-semantic-cache-inbound.mdx)
policy in the app's [policy chain](../ai-gateway/policy-chains.mdx). It keys on
the prompt for you and keeps each app's cached responses apart, and the tokens
its hits save appear as **Saved Tokens** in the Cache analytics.

## Related

- [Semantic Cache policy reference](../policies/semantic-cache-inbound.mdx)
- [Cache at the gateway](./gateway-caching.mdx)
- [Caching overview](./overview.mdx)