# Streaming responses and Server-Sent Events

Zuplo's runtime is built on web-standard APIs, so streaming responses work the
same way they do in any standards-compliant JavaScript environment. You can
return a `ReadableStream` as a response body from a
[custom handler](../handlers/custom-handler.mdx), proxy an upstream streaming
response through the gateway unbuffered, or stream LLM tokens through Zuplo's
[AI Gateway](../ai-gateway/overview.mdx). The runtime supports the full
[Streams API](../programmable-api/web-standard-apis.mdx) — `ReadableStream`,
`WritableStream`, and `TransformStream` — so when a handler returns a `Response`
whose body is a `ReadableStream`, Zuplo streams the bytes to the client as they
become available rather than buffering the entire response.

## Return an SSE response from a custom handler

To emit Server-Sent Events from a Zuplo handler, create a `ReadableStream` that
writes SSE-formatted messages and return it with the `text/event-stream` content
type.

```ts
import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function (request: ZuploRequest, context: ZuploContext) {
  const stream = new ReadableStream({
    start(controller) {
      const encoder = new TextEncoder();
      let count = 0;
      const interval = setInterval(() => {
        count++;
        const message = `data: ${JSON.stringify({ count, time: Date.now() })}\n\n`;
        controller.enqueue(encoder.encode(message));

        if (count >= 5) {
          clearInterval(interval);
          controller.close();
        }
      }, 1000);
    },
  });

  return new Response(stream, {
    headers: {
      "content-type": "text/event-stream",
      "cache-control": "no-cache",
    },
  });
}
```

Each SSE message ends with two newline characters (`\n\n`). The
`text/event-stream` content type tells the client to interpret the response as
an event stream.

## Proxy an upstream streaming response

When your upstream API returns a streaming response — for example, an SSE
endpoint or a chunked JSON stream — you can pass it through the gateway
unbuffered. Return the upstream `Response` directly from your handler without
reading its body:

```ts
import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function (request: ZuploRequest, context: ZuploContext) {
  const upstreamResponse = await fetch("https://api.example.com/events", {
    headers: {
      accept: "text/event-stream",
      authorization: `Bearer ${context.env.UPSTREAM_API_KEY}`,
    },
  });

  // Return the response directly — the body streams through unbuffered
  return upstreamResponse;
}
```

Set `UPSTREAM_API_KEY` in your project's
[environment variables](./environment-variables.mdx).

:::caution{title="Avoid reading the body"}

Calling `.text()`, `.json()`, or `.arrayBuffer()` on the upstream response
consumes the entire `ReadableStream` into memory. If you need to inspect the
body _and_ return it, use `.clone()` first — see
[Safely clone a request or response](../programmable-api/safely-clone-a-request-or-response.mdx).

:::

You can also use the built-in [URL Rewrite](../handlers/url-rewrite.mdx) or
[URL Forward](../handlers/url-forward.mdx) handlers to proxy upstream responses.
These handlers pass the response body through as a stream without buffering.

## Streaming and policies

Policies in the Zuplo [request pipeline](../concepts/request-lifecycle.mdx)
interact with streaming responses differently depending on whether they read the
response body:

- **Policies that do not read the body** — such as
  [Set Headers](../policies/set-headers-outbound.mdx) or
  [Remove Headers](../policies/remove-headers-outbound.mdx) — pass the stream
  through without buffering.
- **Policies that read the body** — such as
  [Replace String](../policies/replace-string-outbound.mdx) or
  [XML to JSON](../policies/xml-to-json-outbound.mdx) — consume the entire
  response into memory before returning a new response. This effectively
  disables streaming for that request.

:::tip{title="Keep streaming intact"}

If streaming performance is critical, place only header-modifying policies on
the outbound pipeline of streaming routes. Avoid outbound policies that inspect
or transform the response body.

:::

**Inbound policies** run before the handler executes and do not affect response
streaming. Authentication, rate limiting, and request validation all complete
before the handler returns a streaming response.

## Stream LLM tokens through the AI Gateway

The [Zuplo AI Gateway](../ai-gateway/overview.mdx) preserves streaming behavior
from LLM providers. When you send a request with `"stream": true`, the gateway
proxies the provider's token-by-token response back to your application as it
arrives.

To call the AI Gateway, copy the **API URL** and **API Key** from your app's
page in the AI Gateway. Append `/v1/chat/completions` to the API URL and send
the app's API key as a bearer token. Name the model as `providerName/model`,
where `providerName` is the provider name configured in your gateway. See
[Universal API](../ai-gateway/universal-api.mdx) for details.

```ts
const response = await fetch(
  "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions",
  {
    method: "POST",
    headers: {
      "content-type": "application/json",
      authorization: `Bearer ${process.env.ZUPLO_AI_GATEWAY_API_KEY}`,
    },
    body: JSON.stringify({
      model: "openai/gpt-6-luna",
      stream: true,
      messages: [{ role: "user", content: "Hello" }],
    }),
  },
);

// The response is an SSE stream of token chunks
const reader = response.body!.getReader(); // body is non-null for streamed responses
const decoder = new TextDecoder();
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  console.log(decoder.decode(value, { stream: true }));
}
```

The AI Gateway applies usage limits, policies, and observability to streamed
requests. Token usage and cost are tracked for each request regardless of
whether streaming is enabled. See
[AI Gateway overview](../ai-gateway/overview.mdx) for more on usage limits and
provider configuration.

## Cache streaming responses

Use [StreamingZoneCache](../programmable-api/streaming-zone-cache.mdx) to cache
`ReadableStream` responses across Zuplo's edge network. Split the stream with
`tee()` to return one copy to the client while caching the other:

```ts
import { StreamingZoneCache, ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function (request: ZuploRequest, context: ZuploContext) {
  const cache = new StreamingZoneCache("sse-cache", context);
  const cacheKey = `stream:${request.url}`;

  const cached = await cache.get(cacheKey);
  if (cached) {
    return new Response(cached, {
      headers: { "content-type": "text/event-stream" },
    });
  }

  const upstream = await fetch("https://api.example.com/events");

  // Only cache successful responses that have a body
  if (!upstream.ok || !upstream.body) {
    return upstream;
  }

  const [forCache, forClient] = upstream.body.tee();

  // TTL is in seconds — cache for 5 minutes
  context.waitUntil(cache.put(cacheKey, forCache, 300));

  return new Response(forClient, {
    headers: { "content-type": "text/event-stream" },
  });
}
```

:::note

Caching is best suited for streams that produce a finite, repeatable result (for
example, a pre-generated report). Infinite or user-specific event streams are
generally not good candidates for caching.

:::

## When to use WebSockets instead

SSE is one-directional: the server pushes events to the client. If you need
bidirectional communication — where the client also sends messages over the same
connection — use Zuplo's [WebSocket Handler](../handlers/websocket-handler.mdx)
instead. WebSockets support full-duplex communication and can be proxied to an
upstream WebSocket server.

## Next steps

- [Function Handler](../handlers/custom-handler.mdx) — build custom request
  handlers
- [Web Standard APIs](../programmable-api/web-standard-apis.mdx) — the full list
  of supported web APIs, including the Streams API
- [StreamingZoneCache](../programmable-api/streaming-zone-cache.mdx) — cache
  streaming responses at the edge
- [Safely clone a request or response](../programmable-api/safely-clone-a-request-or-response.mdx)
  — avoid `body-used` errors when inspecting streamed bodies
- [AI Gateway](../ai-gateway/overview.mdx) — stream LLM tokens with cost
  controls and observability
- [WebSocket Handler](../handlers/websocket-handler.mdx) — bidirectional
  real-time communication