Copy page

Guides

Streaming responses and Server-Sent Events

Zuplo's runtime is built on web-standard APIs, so streaming responses work the same way they do in any standards-compliant JavaScript environment. You can return a ReadableStream as a response body from a custom handler, proxy an upstream streaming response through the gateway unbuffered, or stream LLM tokens through Zuplo's AI Gateway. The runtime supports the full Streams API — ReadableStream, WritableStream, and TransformStream — so when a handler returns a Response whose body is a ReadableStream, Zuplo streams the bytes to the client as they become available rather than buffering the entire response.

Return an SSE response from a custom handler

To emit Server-Sent Events from a Zuplo handler, create a ReadableStream that writes SSE-formatted messages and return it with the text/event-stream content type.

Code
import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function (request: ZuploRequest, context: ZuploContext) {
  const stream = new ReadableStream({
    start(controller) {
      const encoder = new TextEncoder();
      let count = 0;
      const interval = setInterval(() => {
        count++;
        const message = `data: ${JSON.stringify({ count, time: Date.now() })}\n\n`;
        controller.enqueue(encoder.encode(message));

        if (count >= 5) {
          clearInterval(interval);
          controller.close();
        }
      }, 1000);
    },
  });

  return new Response(stream, {
    headers: {
      "content-type": "text/event-stream",
      "cache-control": "no-cache",
    },
  });
}

Each SSE message ends with two newline characters (\n\n). The text/event-stream content type tells the client to interpret the response as an event stream.

Proxy an upstream streaming response

When your upstream API returns a streaming response — for example, an SSE endpoint or a chunked JSON stream — you can pass it through the gateway unbuffered. Return the upstream Response directly from your handler without reading its body:

Code
import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function (request: ZuploRequest, context: ZuploContext) {
  const upstreamResponse = await fetch("https://api.example.com/events", {
    headers: {
      accept: "text/event-stream",
      authorization: `Bearer ${context.env.UPSTREAM_API_KEY}`,
    },
  });

  // Return the response directly — the body streams through unbuffered
  return upstreamResponse;
}

Set UPSTREAM_API_KEY in your project's environment variables.

You can also use the built-in URL Rewrite or URL Forward handlers to proxy upstream responses. These handlers pass the response body through as a stream without buffering.

Streaming and policies

Policies in the Zuplo request pipeline interact with streaming responses differently depending on whether they read the response body:

• Policies that do not read the body — such as Set Headers or Remove Headers — pass the stream through without buffering.
• Policies that read the body — such as Replace String or XML to JSON — consume the entire response into memory before returning a new response. This effectively disables streaming for that request.

Inbound policies run before the handler executes and do not affect response streaming. Authentication, rate limiting, and request validation all complete before the handler returns a streaming response.

Stream LLM tokens through the AI Gateway

The Zuplo AI Gateway preserves streaming behavior from LLM providers. When you send a request with "stream": true, the gateway proxies the provider's token-by-token response back to your application as it arrives.

To call the AI Gateway, copy the API URL and API Key from your app's page in the AI Gateway. Append /v1/chat/completions to the API URL and send the app's API key as a bearer token. Name the model as providerName/model, where providerName is the provider name configured in your gateway. See Universal API for details.

Code
const response = await fetch(
  "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions",
  {
    method: "POST",
    headers: {
      "content-type": "application/json",
      authorization: `Bearer ${process.env.ZUPLO_AI_GATEWAY_API_KEY}`,
    },
    body: JSON.stringify({
      model: "openai/gpt-6-luna",
      stream: true,
      messages: [{ role: "user", content: "Hello" }],
    }),
  },
);

// The response is an SSE stream of token chunks
const reader = response.body!.getReader(); // body is non-null for streamed responses
const decoder = new TextDecoder();
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  console.log(decoder.decode(value, { stream: true }));
}

The AI Gateway applies usage limits, policies, and observability to streamed requests. Token usage and cost are tracked for each request regardless of whether streaming is enabled. See AI Gateway overview for more on usage limits and provider configuration.

Cache streaming responses

Use StreamingZoneCache to cache ReadableStream responses across Zuplo's edge network. Split the stream with tee() to return one copy to the client while caching the other:

Code
import { StreamingZoneCache, ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function (request: ZuploRequest, context: ZuploContext) {
  const cache = new StreamingZoneCache("sse-cache", context);
  const cacheKey = `stream:${request.url}`;

  const cached = await cache.get(cacheKey);
  if (cached) {
    return new Response(cached, {
      headers: { "content-type": "text/event-stream" },
    });
  }

  const upstream = await fetch("https://api.example.com/events");

  // Only cache successful responses that have a body
  if (!upstream.ok || !upstream.body) {
    return upstream;
  }

  const [forCache, forClient] = upstream.body.tee();

  // TTL is in seconds — cache for 5 minutes
  context.waitUntil(cache.put(cacheKey, forCache, 300));

  return new Response(forClient, {
    headers: { "content-type": "text/event-stream" },
  });
}

When to use WebSockets instead

SSE is one-directional: the server pushes events to the client. If you need bidirectional communication — where the client also sends messages over the same connection — use Zuplo's WebSocket Handler instead. WebSockets support full-duplex communication and can be proxied to an upstream WebSocket server.

Next steps

• Function Handler — build custom request handlers
• Web Standard APIs — the full list of supported web APIs, including the Streams API
• StreamingZoneCache — cache streaming responses at the edge
• Safely clone a request or response — avoid body-used errors when inspecting streamed bodies
• AI Gateway — stream LLM tokens with cost controls and observability
• WebSocket Handler — bidirectional real-time communication