Caching in Zuplo
A cached response is one your backend never serves. Every hit removes a database query, a compute cycle, and an egress charge from your bill, and it returns to the caller in a fraction of the time the real work takes.
The question is rarely whether to cache, but where. The same response can be cached in four different places, each with a different reach, a different invalidation story, and a different price when it misses.
The four layers
When Zuplo is in the request path, a response can be held at any of these points. They run in order from the client inward, and each one only pays for itself when the layer in front of it misses.
1. The CDN edge cache is closest to the client and furthest from the data. It lives in a CDN you run in front of the gateway: Akamai, Fastly, Cloudflare, or CloudFront. Zuplo doesn't operate this cache; the CDN Cache Control policy writes the headers your CDN reads to decide how long to hold a response and which purge tags it belongs to. A hit here never reaches your gateway at all, which makes it the only layer that removes the request from your infrastructure entirely. A miss costs a full trip to the gateway plus everything behind it.
2. The gateway response cache is the first thing inside Zuplo. The
Caching policy stores whole responses keyed
by method, URL, query string, and (by default) the Authorization header, and
serves them from an inbound policy. A hit short-circuits the rest of the
pipeline: the handler, the backend, and every outbound policy. A miss costs the
full pipeline and a backend round trip.
3. The programmable cache lives inside your own code, in a handler or a
custom policy, through the Cache API or
ZoneCache. It's the only layer that can
cache something smaller than a response: a shared fragment of a payload, a
tenant lookup, a JWKS document. A miss costs only the work that one cached value
stood in for (one query, one fetch), not the whole request.
4. The backend's own cache is closest to the data and does nothing for the network hops in front of it. By the time a request arrives, every layer above has already paid its latency. A miss here costs the real work: the query, the render, the computation. You build and run this one yourself; you configure the three above it in Zuplo.
Which layer to use
Find the row that matches the endpoint in front of you. The rows are ordered roughly by how much a hit saves.
| Situation | Layer | Mechanism | Where to read |
|---|---|---|---|
| A public catalog endpoint that returns the same body to everyone | CDN edge | cdn-cache-control-outbound with an edge and a client TTL | Cache at the CDN |
| An authenticated endpoint whose body is identical for every caller | CDN edge | cdn-cache-control-outbound with client.visibility: "private" so only the CDN stores it | Cache at the CDN |
| A mostly shared response with a small caller-specific slice | Programmable | Cache the shared fragment with the Cache API or ZoneCache, fetch only the dynamic part | Cache part of a response |
| A response whose cacheability depends on what it contains (empty results, error bodies, flags) | CDN edge, per response | A cacheConfig function that returns TTLs and purge tags for the response in hand | Per-response cache rules |
| Caching rules no built-in policy expresses (a JWT-claim key, a size limit, an upstream TTL) | Programmable | A custom inbound and outbound policy pair built on the Cache API | Build a custom caching policy |
| Identical responses served to many callers, with no CDN in front of the gateway | Gateway | caching-inbound with an expirationSecondsTtl | Cache at the gateway |
| A GraphQL query repeated across callers, where the cache key is the query and its variables | Gateway | graphql-cache-inbound | GraphQL Cache policy |
| An LLM prompt worded differently each time but asking the same question | Gateway | semantic-cache-inbound, which matches on prompt similarity rather than an exact key | Semantic Cache policy |
The intuitive answer is wrong in these cases:
- An authenticated response doesn't have to skip the edge. If the body is
the same for every caller and the
Authorizationheader only decides whether the caller may have it, the CDN can hold one copy while browsers hold none. A singleCache-Controlheader can't express that; a targeted edge header can. - A cache that only holds whole responses can't help a payload split 90% shared, 10% personal. Caching the shared 90% and assembling the rest per request turns a full-size backend call into a small one. That's the pattern with the largest cost delta on most APIs, and it belongs in code.
Combining layers
Put the CDN edge in front of the programmable cache. This is the combination worth reaching for first. The CDN absorbs the requests that are identical across callers, and the programmable cache makes the requests that get through cheap. The handler assembles a response from a cached shared fragment plus one small backend call instead of one full-size call per request. The two layers touch nothing in common: one sets response headers on the way out, the other reads and writes keys inside your handler. A purge at the edge doesn't disturb the programmable cache, and vice versa.
Stacking the gateway response cache with a programmable cache inside your
handler is also safe. A caching-inbound hit skips the handler, so the
programmable cache simply becomes the miss path.
Never stack caching-inbound and cdn-cache-control-outbound
Pick one. A gateway cache hit is served from an inbound policy, and an inbound
policy that returns a response short-circuits the pipeline before any outbound
policy runs, so cdn-cache-control-outbound never executes on a hit. Your CDN
then receives the copy that caching-inbound stored, and caching-inbound
sanitizes every copy it stores: it drops several CDN cache headers (exactly
which ones depends on the CDN) and overwrites Cache-Control with its own
s-maxage (its TTL, 60 seconds by default). The edge and client TTLs collapse
into one number.
The failure is invisible on cache misses. Misses run the full outbound stack and produce exactly the headers you configured, so the route looks correct in testing and starts sending the wrong cache policy to your CDN only once the gateway cache warms up. Nothing logs it.
Where to go next
The table above links a guide for every layer you configure in Zuplo. For the
runtime reference behind the programmable cache, see
ZoneCache for zone-local key-value
storage and the Cache API for whole
Request/Response pairs. Those two also cover the case the table leaves out:
expensive reference data such as configuration, tenant metadata, or a JWKS
document looked up on every request.