---
title: "Load Testing Your API from Multiple Regions"
description: "A load generator in the same cloud as your backend skews every baseline, and a few hosts near one metro can send all of a multi-region deployment's traffic to one region. How to place load generators so a test measures what you think it does."
canonicalUrl: "https://zuplo.com/blog/2026/08/04/load-test-from-multiple-regions"
pageType: "blog"
date: "2026-08-04"
authors: "nate"
tags: "API Gateway, API Best Practices"
image: "https://zuplo.com/og?text=Load%20Testing%20Your%20API%20from%20Multiple%20Regions"
---
A large customer came to us with numbers showing our gateway adding tens of
milliseconds to every request. They had done the responsible thing and built a
direct-vs-gateway comparison: one leg straight to the backend, one leg through
the gateway, same payloads, same auth, same everything. The gateway leg was
clearly slower. It looked like an open-and-shut case against us.
The problem was in a setting nobody had touched. They were driving both legs
from a hosted load-testing platform whose default region happened to sit in the
same cloud as their backend. So the direct leg never left the provider's private
network, while the gateway leg paid real public-internet costs on every request,
out and back. We re-ran the same comparison from neutral bare metal in a third
location and the measured worst-case overhead roughly halved. The gateway had
not changed; the baseline had been measuring a private network path the whole
time.
Nothing about that setup is unusual. It is the default one, and most hosted
platforms keep their load zones in one or two clouds - every public Grafana
Cloud k6 zone runs on AWS - so if your backend is on AWS, the odds are good that
your baseline is already rigged and nobody decided to rig it.
Where your load generators run (i.e. the machines sending the requests) shapes
just about every number a load test produces, and in most tests nobody chose
that location at all. The tool picked a default region, or someone grabbed the
VM nearest the office, and every latency figure in the report inherited that
choice. The [previous post in this series](/blog/how-not-to-load-test-an-api)
covered making the numbers mean something. This one covers the physical half -
where the requests actually come from.
Nothing in the tooling flags this. Generator placement doesn't appear in the
script, the report, or the dashboards, even though it shapes all three. Let's go
through the placement decisions in order.
## Running load generators on a different cloud than your backend
Cloud providers route traffic between their own data centers over private
backbone, and traffic between services in one region often never leaves the
building. A generator sharing a provider with your backend measures a path
almost none of your users will take. The absolute numbers come out flattering,
but the variance gives it away: public-internet latency has jitter, occasional
retransmits, and a visible spread between p50 and p99 (we covered
[reading percentile spread instead of averages](/learning-center/solving-latency-problems-in-high-traffic-apis)
previously), while an intra-cloud path is implausibly flat.
The baseline leg of a direct-vs-gateway comparison gets run from the
load-testing platform's default region, which happens to share a cloud
provider with the backend. The "gateway overhead" that falls out is mostly a
private network being compared against the public internet.
The remedy costs almost nothing: as
[our performance-testing guide](/docs/articles/performance-testing) puts it,
never run performance tests from the same cloud provider as your backend, and
prefer regions where your users are. For an A/B comparison to be fair, run both
legs over the public internet from the same locations with identical payloads
and auth, so the only variable left is what you're comparing.
## Spreading load across regions and resolvers
One region of generators creates a second, less obvious problem once the target
is multi-region. A DNS-based global load balancer typically picks a data center
from where the client's resolver sits, not where the client sits, so each
generator location resolves once and lands on one data center. Run it from a few
locations and you have tested one cluster with extra DNS noise on top.
Anycast-fronted systems get there differently: many locations advertise the same
IP address and
[routers direct each request to the nearest machine](https://blog.cloudflare.com/a-brief-anycast-primer/),
so a given generator location lands on the same PoP (point of presence)
essentially every time, and one city's test measures one PoP's ceiling out of
hundreds.
From the inside this looks exactly like a backend capacity problem, which is why
it wastes so much time. A customer of ours was validating a brand new
two-region, active-active deployment behind a DNS-based global load balancer.
They fired a few thousand RPS at it from a couple of hosts near one metro.
Instances started running out of memory and restarting, error rates climbed, and
the conclusion in the room was that the new architecture couldn't take the load.
The second region was idle the entire time. Between DNS caching, connection
reuse, and location-based routing, every single request had gone to one region -
a region provisioned for its share of the traffic, not all of it. The deployment
had been failed for a test it was never actually given.
The tell was sitting in the DNS data. A multi-thousand-RPS test had produced
only a few hundred lookups, which is the signature of a handful of clients
resolving once and then hammering the one answer they got. Query volume tells
you where your load actually went, and most managed DNS providers expose counts
per zone and per record. Check it on every distributed test - it is one graph
and it settles the argument.
The saturation reached the telemetry too, in a way worth recognizing. The load
tool reported roughly twice as many successful responses as the gateway's
analytics did. Crashed instances explain the gap: an instance that dies never
flushes its telemetry, so the tool had correctly counted successes that no
instance survived to report. The error counts disagreed for a separate reason -
5xx responses from the load balancer in front of the gateway never appear in
gateway analytics at all. Observability tends to degrade first under saturation,
so when two layers stop agreeing on the numbers, that disagreement is itself the
finding.
What to do depends on the question. For realism (production-shaped traffic),
spread generators across regions and resolvers so the load balancer sees a
population like your users. For regional capacity, do the opposite: bypass the
global load balancer and hit each region at the share it's provisioned for. In
k6 that's one script with two scenarios, each pinned to a region's direct
endpoint, driven by an arrival-rate executor (an open workload model, where
requests arrive on a schedule regardless of response times; the
[previous post](/blog/how-not-to-load-test-an-api) covers why that matters).
Each scenario pre-allocates its own pool of VUs (i.e. virtual users, k6's
request-sending workers) and grows it when responses slow:
```javascript
import http from "k6/http";
// Bypass the DNS load balancer: point each scenario at one region's
// direct endpoint, each carrying that region's share of the total load.
const REGION_A = __ENV.REGION_A_URL || "https://region-a.api.example.com";
const REGION_B = __ENV.REGION_B_URL || "https://region-b.api.example.com";
export const options = {
// we only need the timings, and bodies at these rates burn generator memory
discardResponseBodies: true,
// each scenario tags its own metrics, so you can tell the regions apart
scenarios: {
region_a: {
executor: "constant-arrival-rate",
exec: "regionA",
rate: 1000,
timeUnit: "1s",
duration: "10m",
preAllocatedVUs: 200,
maxVUs: 500, // separate pool per scenario, so budget for 1,000 VUs
},
region_b: {
executor: "constant-arrival-rate",
exec: "regionB",
rate: 1000,
timeUnit: "1s",
duration: "10m",
preAllocatedVUs: 200,
maxVUs: 500,
},
},
};
export function regionA() {
http.get(`${REGION_A}/v1/resource`);
}
export function regionB() {
http.get(`${REGION_B}/v1/resource`);
}
```
On a hosted platform, geographic spread is typically a first-class option:
Grafana Cloud k6 distributes a test across zones with a `cloud.distribution`
block in the same `options` object:
```javascript
export const options = {
// ...the scenarios block from above...
cloud: {
distribution: {
us: { loadZone: "amazon:us:ashburn", percent: 50 },
asia: { loadZone: "amazon:jp:tokyo", percent: 50 },
},
},
};
```
Check where a platform's zones physically run before trusting a baseline: a zone
sharing a cloud provider with your backend brings back the earlier problem.
## Sending load from more than one source IP
Spreading across regions usually handles source diversity, but treat IP count as
its own requirement: several systems in the path key on client IP. Rate limiting
is the obvious one: under per-IP limits at the API or the gateway, two generator
hosts look like two wildly abusive clients instead of thousands of normal ones,
and you measure the rate limiter rather than the API. WAF and bot-mitigation
rules score by IP too, and one address at thousands of RPS is the exact profile
they exist to stop.
Load balancers add a subtler version. Many of them pool connections from a small
set of source IPs, so a two-host test can be funneled through less of the load
balancer than production traffic would. Source-IP hashing (affinity schemes
where the same client always reaches the same backend) does the same one layer
down, concentrating a two-IP test onto one or two backend nodes, so your numbers
describe those nodes, not the fleet.
The fix is more generator hosts, each with its own address, spread across those
regions. Hosted platforms typically fan out over many machines and IPs, but
confirm rather than assume. And if a WAF or bot-mitigation layer sits in the
path, tell its operator a test is coming so their automation doesn't decide for
you.
## Monitoring the load generator as part of the system under test
Load generators tend to fail quietly. An overloaded backend throws errors; an
overloaded generator sends less load than you asked for, still produces a
normal-looking report, and its numbers drift optimistic because fewer requests
in flight means less queueing at the target. The fleet has CPU, memory, file
descriptors, an ephemeral port range, and a network path of its own, any of
which can become the run's limiting factor.
The one that catches people out is the connection budget. The TLDR is that a
generator host can only hold so many connections open at once, and once you run
out, the test starts behaving as though the API were throttling you: requests
stall, connections get refused, and throughput flattens at a ceiling that
nothing on the server side explains. It is a miserable thing to diagnose
precisely because every symptom points at the target.
Two limits produce it. Every outbound connection consumes an ephemeral port, and
the default Linux range, 32768 to 60999, provides 28,232 of them;
[Cloudflare's write-up on ephemeral port exhaustion](https://blog.cloudflare.com/how-to-stop-running-out-of-ephemeral-ports-and-start-to-love-long-lived-connections/)
has the details. High rates without connection reuse churn through that range,
and connections in TIME_WAIT keep ports out of circulation after their requests
finish. Separately, anything doing NAT or stateful filtering in front of the
generators records every connection in a tracking table of finite size
(`conntrack` on Linux), and once that table is full new connections get dropped
rather than queued. An AWS NAT gateway supports up to
[55,000 simultaneous connections to each unique destination](https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-basics.html)
per IP address, and since a load test is a great many connections to a single
destination, a whole fleet behind one NAT gateway shares that one cap.
The practical answer is usually not to solve this yourself. A hosted platform
like Grafana Cloud k6 spreads a run across enough machines and addresses that no
single host approaches either ceiling, and that distribution is a good part of
what you are paying for. If you do run your own generators, add hosts rather
than tuning `sysctl` values on a few of them, and treat an unexplained
throughput ceiling as the generator's fault until you have ruled it out.
How fast you get there depends on connection reuse, and tools differ. Apache's
[ab defaults to keep-alive off](https://httpd.apache.org/docs/2.4/programs/ab.html),
opening a fresh connection per request, while k6 reuses connections unless you
set
[noConnectionReuse](https://grafana.com/docs/k6/latest/using-k6/k6-options/reference/).
Two tools reporting the same RPS can therefore exercise completely different
connection loads, so know which yours does before comparing across tools.
The
[k6 guide to running large tests](https://grafana.com/docs/k6/latest/testing-guides/running-large-tests/)
recommends keeping the generator's CPU below 80% and memory below 90%, and the
reasoning applies to any tool: past that point it distorts timings long before
anything visibly fails. k6 also has a built-in tell:
[dropped_iterations](https://grafana.com/docs/k6/latest/using-k6/scenarios/concepts/dropped-iterations/)
increments whenever an arrival-rate scenario can't start an iteration on
schedule because no VU is free. A nonzero value means the test is no longer
sending the load the script describes, so wire it into a threshold and let the
run fail itself:
```javascript
export const options = {
// ...the scenarios block from above...
thresholds: {
dropped_iterations: ["count==0"],
// timings from a run that was mostly errors aren't worth reading
http_req_failed: ["rate<0.01"],
},
};
```
Headroom in `maxVUs` keeps that threshold green, since the VUs an arrival rate
needs scale with response time; size it for the degraded case you want to
observe, not the happy case. If iterations still drop with generous headroom and
healthy CPU, that is typically the finding - holding the rate needs more
capacity than the fleet has, so add generator machines rather than loosen the
threshold.
## Decide where your generators run on purpose
Both stories in this post end the same way. A team ran a test, got a bad result,
and drew a conclusion about their system - when what they had actually measured
was where they happened to be sending traffic from. One team nearly filed a
performance regression against a gateway. The other nearly re-architected a
deployment that was working.
The checklist that would have caught both is short enough to run before every
distributed test:
- Generators in a **different cloud** than the backend, and both legs of any A/B
comparison over the same path.
- **More than one region**, chosen to match where your users are.
- **More than one resolver and source IP**, so rate limiters, WAFs, and
source-IP hashing see a population rather than two very rude clients.
- **DNS query counts checked** against the request count after the run.
- **Generator CPU, memory, connection limits, and dropped iterations** on the
same dashboard as everything else.
Placement is only half of the problem, though. Every intermediary in the path is
the other half: load balancers, CDNs, WAFs, and gateways all reshape the result
before the origin ever sees it, which is
[the next post in this series](/blog/load-testing-gateways-and-cdns).
And if the test you're planning points at a Zuplo gateway, get in touch before
you run it. We can allowlist your generator IPs, tell you which of our regions
you're actually hitting, and watch the run from our side while it happens.