---
title: "Load Testing Through Proxies, Gateways, and CDNs"
description: "A full-path load test measures every proxy, gateway, CDN, and load balancer in front of your API. Learn how to budget latency, warm the path, validate rate limits, control caching, and identify the failing layer."
canonicalUrl: "https://zuplo.com/blog/2026/08/11/load-testing-gateways-and-cdns"
pageType: "blog"
date: "2026-08-11"
authors: "nate"
tags: "API Gateway, API Best Practices, API Rate Limiting"
image: "https://zuplo.com/og?text=Load%20Testing%20Through%20Proxies%2C%20Gateways%2C%20and%20CDNs"
---
Almost no production API is reachable directly. A request usually crosses DNS, a
CDN, a WAF, a load balancer, and an API gateway before the origin sees it. A
load test measures all of them, and it tends to find the weakest layer first.
That is exactly why you should test the production path. It is also why blaming
the origin, or the gateway, from one end-to-end number is mostly guesswork.
## Write down the expected cost of every layer
Before the test, document each hop and the latency it is expected to add.
Include the conditions behind the number: region, policies, cache state,
connection reuse, and whether the node is warm.
We publish our own numbers. A Zuplo gateway adds about 20 to 30ms of base
latency with no policies. Most policies add 1 to 5ms, while policies that call
an external service generally add 5 to 15ms. A cold request to a fresh edge node
can be 100 to 200ms slower. Our
[performance-testing guide](/docs/articles/performance-testing) documents the
baseline and testing conditions.
Those are planning numbers, not universal constants. Get the equivalent from
every provider in your path, then add the expected costs into one end-to-end
budget.
This budget does not prove which layer caused an outlier. It tells you whether
the end-to-end result is plausible and gives each owner a number to validate.
Without it, a latency chart has no baseline.
## Warm the path before collecting steady-state data
Edge and serverless platforms start with little or no capacity in a cold
location. The first requests can pay for runtime initialization, configuration
loading, and connection setup. A test that jumps from zero to peak traffic mixes
startup behavior into its steady-state percentiles.
Use a separate warm-up scenario and exclude its samples from thresholds. The
following k6 configuration sends 20 RPS for two minutes before starting the
500-RPS measurement window:
```javascript
export const options = {
discardResponseBodies: true,
scenarios: {
warmup: {
executor: "constant-arrival-rate",
rate: 20,
timeUnit: "1s",
duration: "2m",
preAllocatedVUs: 20,
startTime: "0s",
},
measurement: {
executor: "constant-arrival-rate",
rate: 500,
timeUnit: "1s",
duration: "5m",
preAllocatedVUs: 200,
maxVUs: 500,
startTime: "2m",
},
},
};
```
k6 tags metrics with the scenario name. Use that tag in every threshold so the
warm-up data stays out of the measurement.
In our own seven-minute runs against a trivial origin, the gateway added about
40ms end to end after the edge had warmed. The first couple of minutes were
higher while nodes absorbed the step-function ramp. That gap matters: the 40ms
includes the extra network hop, while the published base number measures gateway
processing. Mixing those two numbers would create an argument where there is no
disagreement.
Warm-up is not permission to hide cold starts. Measure cold-start behavior in a
separate test if it matters to your users. Keep the two questions separate:
steady-state capacity and latency after a scale-to-zero event are different
properties.
## Treat expected 429 responses as expected
A rate-limit test should cross the configured limit. If it never receives a 429,
the limiter may not be enforcing the policy. If the script counts every 429 as
an unexpected failure, a working limiter makes the test fail.
In k6, declare 429 as an expected response and count it separately:
```javascript
import http from "k6/http";
import { Counter } from "k6/metrics";
const BASE_URL = __ENV.TARGET_URL || "https://api.example.com";
http.setResponseCallback(http.expectedStatuses({ min: 200, max: 299 }, 429));
const rateLimited = new Counter("rate_limited");
export default function () {
const res = http.get(`${BASE_URL}/products`);
if (res.status === 429) {
rateLimited.add(1);
}
}
```
Add scenario-specific thresholds to the `options` object:
```javascript
thresholds: {
"http_req_failed{scenario:measurement}": ["rate<0.01"],
"rate_limited{scenario:measurement}": ["count>0"],
"http_req_duration{scenario:measurement}": ["p(95)<300", "p(99)<800"],
},
```
The callback keeps expected 429s out of `http_req_failed`. The custom counter
makes the run fail if the limiter never activates.
429s still contribute to `http_req_duration`, and they are often faster than
successful responses. Keep the
[status-specific latency trends](/blog/how-not-to-load-test-an-api) from the
first article so quick rejections do not improve the reported latency.
For Zuplo's [inbound rate-limit policy](/docs/policies/rate-limit-inbound), the
configured mode changes the request-path cost:
| `mode` | Request-path behavior | Trade-off |
| ------------------ | ------------------------------------------- | ---------------------------------- |
| `strict` (default) | Waits for the external count before serving | Accurate distributed enforcement |
| `async` | Updates the count outside the critical path | Some requests can exceed the limit |
Use strict counting when the limit protects billing or contractual usage. An
asynchronous limit can be appropriate when the goal is protecting infrastructure
and a small overshoot is acceptable.
This is not theoretical for us. A sustained load test exposed an unhandled
exception in our rate-limiter client when its backing service responded slowly.
Our functional tests had never driven that error path hard enough to find it. We
fixed the client, but the more useful lesson was that a full-path load test
audits the defensive machinery too. Testing around the limiter would have missed
the bug entirely.
Avoid sub-second windows for a distributed rate limiter. At that scale, the test
can become a measurement of synchronization delay rather than the policy you
intended to validate.
## Define the cache hit ratio in the test design
Every request through a cache is either a hit or a miss. The ratio changes
latency, origin load, intermediary work, and often rate-limit behavior.
A script that repeatedly requests one URL usually creates a nearly 100% hit
ratio after the first request. That may be useful for measuring cached delivery,
but it tells you little about origin capacity. Randomizing every key creates the
opposite problem.
Choose keys from a production-like distribution. Report the measured hit ratio
next to the latency and throughput results. If you need both extremes, run
separate cache-hot and cache-cold scenarios instead of blending them.
## Map every hop and keep evidence from every layer
Draw the full request path before the first request. Include components that are
easy to overlook:
- DNS proxies and secondary CDN accounts
- WAF and bot-management services
- Public and private load balancers
- NAT gateways and egress proxies
- API gateways and service-mesh proxies
- The origin platform in front of application code
For each layer, record request count, status, latency, and request or trace ID
when available. If one layer logs a request and the next layer does not, you
have narrowed the loss to one segment of the path.
Do not tear down test infrastructure immediately after the run. Keep the
generators, dashboards, logs, and target environment available until the team
has finished investigating. A failed run without its evidence is just an
expensive retry.
Providers may also require advance notice. DDoS and bot systems are designed to
block high request rates from a small set of IPs. Review each provider's policy,
book the test window, and allowlist generator addresses. Akamai publishes
[load-testing guidance](https://community.akamai.com/customers/s/article/Best-Practices-for-Load-Testing-with-Akamai-CDN?language=en_US),
and AWS distinguishes permitted testing from
[simulated DDoS events](https://aws.amazon.com/ec2/testing/).
Confirm the test window and generator IP allowlist with every provider in the
request path. Otherwise the test may benchmark a block page instead of the
API.
## Make the origin behave like the real service
A hello-world origin hides concurrency limits. Little's law says that in-flight
requests equal arrival rate multiplied by average response time. At 30,000 RPS,
a 400ms origin creates roughly 12,000 in-flight requests. A 5ms mock creates
about 150.
Those workloads put very different pressure on connection pools, load balancers,
gateways, and NAT tables.
Give the test origin a configurable delay and set it from production latency, or
replay a realistic response-time distribution. Add a degraded scenario with
multi-second responses to expose interactions between timeouts, retries, and
queue shedding.
## Full-path load test checklist
Before the run:
1. Draw every hop from generator to origin.
2. Record the expected overhead and operating conditions for each layer.
3. Confirm the provider test window and allowlist generator IPs.
4. Add a warm-up phase and exclude it from steady-state thresholds.
5. Define expected status codes, including 429s.
6. Define and measure the cache hit ratio.
7. Use an origin with realistic latency and failure behavior.
8. Keep the environment and telemetry available for investigation.
A full-path test should measure the full stack. We publish our overhead so you
can hold us to it, and we would rather help set up a clean test than argue about
a dirty result afterward. When the result is wrong, the next job is attribution.
The following article shows how to use
[logs and distributed traces to find the responsible layer](/blog/load-testing-with-tracing).