API Analytics and Observability: Metrics, Alerts, and Owning Your Telemetry

Every API platform ships an analytics dashboard. They all show roughly the same four charts: request volume, latency, error rate, and top endpoints. The dashboards are not where teams get into trouble.

Teams get into trouble at the three places past the dashboard: deciding which of those numbers deserves to wake someone up, getting from a spike on a chart to the specific failing request, and — the one that surfaces eighteen months in — getting the underlying data out of the vendor’s store and into the warehouse where the rest of the business lives.

This guide walks those three problems in order, then covers what it takes to treat API telemetry as data you own rather than a feature you rent. If you want the foundational material first — the three pillars, structured logging patterns, how tracing works — start with the complete guide to API observability and monitoring and come back here.

• The metrics that actually matter
  • Latency, as percentiles
  • Error rate, split by who caused it
  • Traffic and adoption
  • The dimensions matter more than the metrics
• Where alerting actually belongs
  • Alert on symptoms, not causes
  • Synthetic checks catch what traffic-based alerts cannot
• From a spike to a root cause
  • Make every request findable
  • Attach the dimensions you will want to pivot on
• Owning your telemetry: export versus stream
  • The export model
  • The streaming model
  • Which one you want
• Doing this with Zuplo
  • What is built in
  • Forwarding metrics
  • Forwarding logs
  • Traces over OTLP
  • Custom signals in TypeScript
• A practical rollout order
• What to ask during evaluation

The metrics that actually matter

An API produces an enormous number of measurable things. Four of them carry most of the signal.

Latency, as percentiles

Average latency is close to useless. A p50 of 40ms tells you nothing about the customer whose integration times out, because averages hide the tail where all the pain lives. Track p50, p95, and p99 — p50 for the typical experience, p95 for the SLO you commit to, p99 for the complaints you will receive.

Measure latency at the edge, not at your backend. The number that matters is what the consumer experiences: DNS, TLS, gateway processing, backend time, and the return trip. A backend that reports 20ms while consumers see 400ms is a routing or geography problem you will never find in backend metrics alone.

Split gateway time from upstream time wherever your tooling allows. When p99 climbs, the first question is always “is this us or is this the backend?” and a single combined number cannot answer it.

Error rate, split by who caused it

An aggregate error rate mixes two unrelated stories. 4xx responses are usually consumers doing something wrong — a bad payload, an expired key, a rate limit hit. 5xx responses are you doing something wrong. Track them separately, alert on them separately.

Within 4xx, 401 and 429 deserve their own lines. A sudden 401 spike usually means a credential rotation went badly or a key expired for a large consumer. A 429 spike means either an abusive client or a rate limit that no longer matches how consumers actually use the API. Both are actionable within minutes if you can see them, and invisible if they are averaged into a single “errors” number.

Traffic and adoption

Request volume by route tells you which endpoints justify engineering investment. Unique active consumers over time tells you whether the API is growing. First-call-to-second-call conversion — the share of newly issued API keys that ever make a second request — is one of the sharpest signals of developer experience problems there is, and almost nobody tracks it.

These are dashboard metrics, not alert metrics. Alerting on volume is how you get paged at 3am on a public holiday because traffic dropped exactly as it should have.

The dimensions matter more than the metrics

The metrics list is short and mostly settled. What separates a useful analytics setup from a decorative one is the dimensions attached to each data point: which route, which consumer or API key, which region, which API version, which deployment build.

Without dimensions, “error rate is 3%” is a dead end. With them, it becomes “one consumer on API version 2 in eu-west is getting 100% 500s on one route since build 4f2a” — which is a fix, not an investigation. Deciding these dimensions up front matters, because you cannot retroactively add a dimension to data that has already been recorded. Per-consumer attribution in particular is worth wiring on day one; see tracking API performance per customer for why aggregates fall short.

Where alerting actually belongs

Here is a question worth asking any API management vendor during evaluation: does your platform send alerts, or does it expect my monitoring platform to?

Both answers are defensible. Neither is universal, and assuming the wrong one is how teams end up with a dashboard nobody watches and no page when it matters.

The case for alerting living in your existing monitoring platform is strong: it is where your on-call rotation, escalation policies, maintenance windows, and SLO definitions already are. An API gateway that ships its own parallel alerting system asks you to maintain two of everything, and to accept that a gateway alert cannot be correlated with the database saturation that caused it.

The case against is equally real: if the gateway does not alert, you cannot get value from its telemetry until you have wired up forwarding to something that does. That is real setup work on day one, and it is worth knowing about before you commit.

Alert on symptoms, not causes

Whichever platform hosts the rules, the good ones look the same. Alert on a small number of symptoms that consumers feel:

• p95 latency above your SLO for 5 minutes
• 5xx rate above a threshold over a rolling window
• Availability dropping below target
• Saturation on any resource you own — connection pools, queue depth, memory

Cause-level signals — cache hit ratio dropping, a specific fault code increasing, one upstream getting slower — belong on dashboards and in runbooks. They are what you look at after a symptom alert fires. Promoting them to pages is the fastest route to alert fatigue, and a team that has learned to ignore alerts is worse off than a team with none.

Use multi-window rules where you can: a fast-burn condition for outages, a slow-burn condition for gradual degradation. A single static threshold either misses slow decay or fires constantly during normal variance.

Synthetic checks catch what traffic-based alerts cannot

Traffic-based alerting has a blind spot: it cannot detect an API that stops receiving requests entirely, because a DNS failure or a certificate expiry produces no traffic to alert on. Synthetic monitoring — a scheduled request from outside your infrastructure that asserts on status and response shape — closes that gap.

Zuplo’s own docs recommend exactly this pattern, suggesting external monitors such as Checkly or Datadog Synthetics against health check endpoints for each network configuration. Have your health check verify its critical dependencies rather than returning a bare OK, or you will get a green check from a gateway whose database is unreachable.

From a spike to a root cause

An alert fires. The clock starts. Everything that happens next depends on decisions you made before the incident.

Make every request findable

Every request through your gateway should carry a unique identifier that appears in the response headers, in the gateway log entry, and in your backend logs. When a customer emails “request at 14:32 failed,” that identifier is the difference between a five-minute lookup and an afternoon of guessing.

Zuplo assigns one automatically and exposes it as the zp-rid header, and it is included on every log entry. Return it to consumers, ask for it in your support templates, and propagate it to your backend so a single search spans both systems.

Attach the dimensions you will want to pivot on

Investigation is pivoting: filter to the failing route, group by consumer, split by region, compare against the previous build. Each of those pivots requires a dimension that was recorded at request time.

In Zuplo, you attach these with context.log.setLogProperties, which adds properties to every subsequent log entry for that request:

import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function policy(
  request: ZuploRequest,
  context: ZuploContext,
) {
  const url = new URL(request.url);

  context.log.setLogProperties!({
    consumerId: request.user?.sub ?? "anonymous",
    apiVersion: url.pathname.split("/")[1] ?? "unknown",
    tenantId: request.headers.get("x-tenant-id") ?? "none",
  });

  return request;
}

Register it as a custom-code-inbound policy on the routes you care about, and every subsequent log line for that request — including ones written by policies further down the chain — carries the same properties. Put it early in the inbound chain, since anything logged before it runs will not have them. Keep the values small: identifiers cost far less than request bodies, and log volume is a real cost on a high-throughput API. Never put credentials, tokens, or personal data in log properties.

The same discipline applies to the fault taxonomy. A 500 that says only “Internal Server Error” tells you nothing at 2am. A structured error log with an error class, the upstream that failed, and the elapsed time before failure lets you group failures by cause and find the one that accounts for 90% of them.

Owning your telemetry: export versus stream

Everything above assumes you can get at your data. That assumption is worth checking, because API platforms differ sharply here and the difference only becomes visible when you need it.

The export model

In the export model, the platform collects telemetry into its own analytics store, gives you dashboards and a report API over that store, and offers an export path to move a copy somewhere else.

Apigee is the well-documented example. Its analytics live in Google’s store; custom reports run against it; and getting the underlying data into your own environment means configuring a datastore that points at BigQuery or Cloud Storage and running an export, which executes asynchronously in the background. For large result sets there is a separate asynchronous custom reports API because interactive queries can time out on big time ranges.

This model works, and Apigee’s implementation is mature. The costs are structural rather than defects:

• Exports are a scheduled or triggered job, so your warehouse copy trails the dashboard rather than matching it.
• Export operations are quota-limited and subject to their own permission model, so the pipeline needs monitoring of its own.
• The report API is vendor-specific, so the queries and jobs you build around it do not move to another platform.
• Analytics availability and export capability can be tier-dependent — worth confirming against current documentation for your specific plan before you design around it.

The streaming model

In the streaming model, the gateway is instrumented with OpenTelemetry and emits metrics, logs, and traces continuously to endpoints you control. There is no vendor store in the middle and no export job, because the data was never exclusively somewhere else.

The practical consequences: telemetry arrives in seconds rather than on an export schedule; the receiving side is a collector you configure, so you can fan the same stream out to a monitoring platform and a warehouse simultaneously; retention is your decision; and the instrumentation is a standard, so switching gateways does not mean rewriting your dashboards and alert rules.

The tradeoff is real too. You now own a collector, or you pay a vendor to host one. Cardinality management becomes your problem — adding a high-cardinality attribute to every metric is an expensive mistake you make once. And you need a backend before the data is useful, whereas a built-in analytics store is useful the moment traffic flows.

Which one you want

Small team, no existing observability platform, want dashboards today: built-in analytics wins, and arguing otherwise is dogma.

Existing observability investment, compliance-driven retention requirements, analytics that need to join API data with billing or product data, or an explicit desire to avoid platform lock-in: the streaming model wins, and the gap widens as the API portfolio grows.

Most mature teams end up running both — a gateway-native view for triage, because it already understands routes and consumers, and a streamed copy in their own stack for correlation and long-horizon analysis.

Doing this with Zuplo

Zuplo is built around the streaming model, with enough built in that you are not blocked on day one.

What is built in

The portal’s Observability section provides an analytics view — request volume, latency percentiles, error rates, consumer breakdowns and geographic distribution — and a live log view where every request lands, searchable and filterable by severity, method, status, and route. Zuplo runs across 300+ edge locations, so latency is measured at the edge rather than at your backend, which means gateway and upstream time are both included. Note that this still starts when the request enters the gateway: client-side DNS, the TLS handshake, and the client-to-edge network leg are outside what the gateway can see, so pair it with a synthetic check if you need true end-to-end numbers. Analytics and log retention windows are plan-dependent, so check what your tier gives you before you rely on a long lookback.

One thing to know going in: Zuplo does not currently ship threshold alerting. Alert rules live in whatever platform you forward telemetry to, plus external synthetic monitoring for availability. Both appear on Zuplo’s public roadmap — threshold alerting as Notifications, and multi-region synthetic checks as API Monitoring — but today, plan on defining rules downstream.

Forwarding metrics

Metrics plugins send request latency, request content length, and response content length to Datadog, Dynatrace, New Relic, or any OTLP endpoint. The OpenTelemetry variant:

import {
  RuntimeExtensions,
  OTelMetricsPlugin,
  environment,
} from "@zuplo/runtime";

export function runtimeInit(runtime: RuntimeExtensions) {
  runtime.addPlugin(
    new OTelMetricsPlugin({
      url: "https://otel-collector.example.com:4318/v1/metrics",
      headers: {
        Authorization: `Bearer ${environment.OTEL_API_KEY}`,
      },
      attributes: {
        "service.name": "my-api",
        "deployment.environment": environment.ENVIRONMENT ?? "development",
      },
      metrics: {
        latency: true,
        requestContentLength: true,
        responseContentLength: true,
      },
      include: {
        statusCode: true,
        httpMethod: true,
        country: false,
        path: false,
      },
    }),
  );
}

The include block is where cardinality discipline happens. Every attribute you turn on multiplies the number of time series your backend stores, and path on an API with many distinct routes is the usual way that bill gets surprising. Turn on what you will actually group by.

Forwarding logs

Logging plugins forward structured logs to AWS CloudWatch, Datadog, Dynatrace, Google Cloud Logging, Loki, New Relic, Splunk, Sumo Logic, or VMware Log Insight. Every entry carries default fields including requestId, environment, environmentType, environmentStage, and buildId — that last one is what lets you attribute a regression to a specific deployment.

If your destination is not on the list, you can write a custom logging plugin, and there are storage-oriented plugins for streaming request records to Azure Blob Storage, Azure Event Hubs, and Hydrolix when the destination is a data platform rather than a log search tool.

Traces over OTLP

The OpenTelemetry plugin instruments the full request lifecycle — inbound policies, handler, outbound policies, and fetch subrequests — and supports W3C trace context propagation so a trace continues into your backend services.

import { OpenTelemetryPlugin } from "@zuplo/otel";
import { RuntimeExtensions, environment } from "@zuplo/runtime";

export function runtimeInit(runtime: RuntimeExtensions) {
  runtime.addPlugin(
    new OpenTelemetryPlugin({
      exporter: {
        url: "https://otel-collector.example.com/v1/traces",
        headers: {
          "api-key": environment.OTEL_API_KEY,
        },
      },
      service: {
        name: "my-api",
        version: "1.0.0",
      },
    }),
  );
}

One subtlety: configuring exporter sends traces to your backend instead of Zuplo’s built-in storage, not in addition to it. To keep the portal view while also streaming out, use the spanProcessors array with a BatchTraceSpanProcessor wrapping an OTLPSpanExporter and a second one wrapping a ZuploSpanExporter.

Traces are stored by Zuplo and viewable in the portal on every plan; exporting them to your own OpenTelemetry backend is an enterprise add-on, as are the logging and metrics plugins, with trial access for development and testing. Sampling is configured through sampling.headSampler.ratio — Zuplo’s default project setup traces everything in a working copy and samples other environments at 10%.

Custom signals in TypeScript

The signals above are the generic ones every API has. The ones that tell you whether your business is healthy are specific to your API, and because Zuplo policies and runtime hooks are TypeScript you can emit them directly rather than inferring them from HTTP status codes later:

import { RuntimeExtensions } from "@zuplo/runtime";

export function runtimeInit(runtime: RuntimeExtensions) {
  runtime.addResponseSendingFinalHook((response, request, context) => {
    if (response.status >= 500) {
      context.log.error(
        {
          event: "upstream_failure",
          status: response.status,
          route: new URL(request.url).pathname,
        },
        "Gateway returned a server error",
      );
    }
  });
}

One implementation detail worth knowing here: custom outbound policies only run when the response is 2xx (response.ok === true), so a 500 skips them entirely. That is exactly why this lives in addResponseSendingFinalHook, which fires just before every response is sent regardless of status and is intended for logging and analytics rather than modifying the response. If you only need to catch failures from one specific backend, a custom handler that checks response.ok after its own fetch works too.

A stable event key plus structured properties is what makes this queryable downstream. Whatever backend receives it can count upstream_failure by route without anyone writing a log-parsing regex.

A practical rollout order

You do not need all of this at once, and teams that try usually stall. A sequence that works:

1. Turn on identity. Require API keys or JWTs so every request has a consumer attached. Without this, every later metric is anonymous and no dimension you add can be joined to a customer.
2. Use the built-in views. Watch the analytics dashboard and log stream for a few weeks. Learn what normal traffic looks like before you write a threshold, because a threshold guessed in advance is a threshold you will tune three times.
3. Add structured log properties. Attach the dimensions you will want to pivot on — consumer, tenant, API version — before the incident that needs them.
4. Forward metrics and logs to your platform. Now define alert rules on p95 latency and 5xx rate, plus a synthetic check for availability.
5. Add tracing. Once alerts work, tracing is what turns “the API is slow” into “this specific upstream call is slow.” Sample it.
6. Stream a copy to your warehouse. When product and finance start asking questions your monitoring platform is bad at answering, fan the same OpenTelemetry stream out to a warehouse rather than building a second pipeline.

The order matters more than the speed. Each step makes the next one useful, and skipping to step six leaves you with a warehouse full of data nobody can attribute to a customer.

What to ask during evaluation

The dashboards will look similar whichever platform you pick. The questions worth asking during evaluation are the ones underneath: where do alert rules live, what does it take to get from a spike to a specific failing request, and what does moving your telemetry into your own stack actually require — a standard protocol, or a vendor-specific export job with its own quotas?

Explore Zuplo’s observability features to see the analytics, logs, and tracing views, or sign up for free and have a gateway emitting OpenTelemetry into your own stack this afternoon.