# Cookbook: Proxy the OpenAI Realtime API

> **Scope of this page**
>
> This page documents the Zuplo **gateway** runtime (API, AI, and MCP gateways), not the Developer Portal.
>
> It applies only to Managed Edge hosting. It doesn't apply to Managed Dedicated or Self-Hosted, so don't use it for those deployments. See [Hosting options](https://zuplo.com/docs/articles/hosting-options).
>
> It requires the Enterprise add-on.

The [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime)
streams speech and text over a WebSocket. Server-side clients authenticate with
an `Authorization: Bearer` header that carries an OpenAI API key. This cookbook
puts a Zuplo route in front of that WebSocket so that:

- Clients authenticate with a Zuplo API key, and your OpenAI key stays on the
  gateway.
- The gateway sets the model in the backend URL, so the model a client puts in
  its own URL has no effect.
- A message policy ends any session in which the client sends its own
  instructions or a `system` message.

<Diagram height="h-48">
  <DiagramNode id="client">Client</DiagramNode>
  <DiagramNode id="zuplo">Zuplo route</DiagramNode>
  <DiagramNode id="openai">OpenAI Realtime API</DiagramNode>
  <DiagramEdge
    from="client"
    to="zuplo"
    label="Authorization: Bearer zpka_..."
  />
  <DiagramEdge from="zuplo" to="openai" label="Authorization: Bearer sk-..." />
</Diagram>

The route uses the
[WebSocket Pipeline Handler](../handlers/websocket-pipeline-handler.mdx), which
is an Enterprise feature.

## Prerequisites

- A Zuplo project on Managed Edge.
- An OpenAI API key with access to a Realtime model.
- An [API Key Authentication](../policies/api-key-inbound.mdx) consumer for each
  client that connects through the gateway.

## 1. Store the OpenAI key and model

Add two [environment variables](./environment-variables.mdx) to your project:

| Name                    | Value                                                  | Secret |
| ----------------------- | ------------------------------------------------------ | ------ |
| `OPENAI_API_KEY`        | Your OpenAI API key                                    | Yes    |
| `OPENAI_REALTIME_MODEL` | The Realtime model clients use, such as `gpt-realtime` | No     |

## 2. Add the policies

Add an API Key policy to authenticate clients, and a Set Headers policy that
replaces the client's `Authorization` header with your OpenAI key. Add both to
`config/policies.json`:

```json title="config/policies.json"
{
  "policies": [
    {
      "name": "api-key-inbound",
      "policyType": "api-key-inbound",
      "handler": {
        "export": "ApiKeyInboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "allowUnauthenticatedRequests": false
        }
      }
    },
    {
      "name": "set-openai-key",
      "policyType": "set-headers-inbound",
      "handler": {
        "export": "SetHeadersInboundPolicy",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "headers": [
            {
              "name": "Authorization",
              "value": "Bearer $env(OPENAI_API_KEY)"
            }
          ]
        }
      }
    }
  ]
}
```

The API Key policy runs first and reads the client's Zuplo key from the
`Authorization` header. The Set Headers policy then overwrites that header, so
OpenAI receives your OpenAI key and never sees the client's key.

## 3. Write the message policy

Create `modules/realtime-policies.ts`. This inbound policy reads each event the
client sends. Clients can change instructions in several ways: `session.update`,
`response.create` with its own `instructions`, or a `conversation.item.create`
that adds a `system` message. The policy checks every event for an
`instructions` field or a `system`-role item at any depth. When it finds one, it
closes the client connection, and the handler then closes the connection to
OpenAI. Every other event passes through unchanged.

```ts title="modules/realtime-policies.ts"
import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

// True when any object in the event sets instructions or adds a system message
function overridesInstructions(value: unknown): boolean {
  if (Array.isArray(value)) {
    return value.some(overridesInstructions);
  }
  if (value !== null && typeof value === "object") {
    const obj = value as Record<string, unknown>;
    if ("instructions" in obj || obj.role === "system") {
      return true;
    }
    return Object.values(obj).some(overridesInstructions);
  }
  return false;
}

export async function protectInstructions(
  data: string,
  target: WebSocket,
  source: WebSocket,
  request: ZuploRequest,
  context: ZuploContext,
) {
  // Forward anything that isn't a JSON event unchanged
  let event: unknown;
  try {
    event = JSON.parse(data);
  } catch {
    return data;
  }

  if (overridesInstructions(event)) {
    context.log.warn("Client tried to replace the instructions", {
      consumer: request.user?.sub,
    });
    source.send(
      JSON.stringify({
        type: "error",
        error: { message: "This gateway doesn't allow custom instructions." },
      }),
    );
    // `source` is the client connection in an inbound policy
    source.close(4003, "Custom instructions aren't allowed");
    return undefined; // don't forward the event to OpenAI
  }

  return data;
}
```

## 4. Add the route

Add a `GET` route that uses the WebSocket Pipeline Handler. The `rewritePattern`
sets the model from the environment variable, so the model a client puts in its
own URL has no effect.

```json title="config/routes.oas.json"
"/v1/realtime": {
  "x-zuplo-path": {
    "pathMode": "open-api"
  },
  "get": {
    "summary": "OpenAI Realtime API",
    "operationId": "openai-realtime",
    "x-zuplo-route": {
      "corsPolicy": "none",
      "handler": {
        "export": "webSocketPipelineHandler",
        "module": "$import(@zuplo/runtime)",
        "options": {
          "rewritePattern": "wss://api.openai.com/v1/realtime?model=${env.OPENAI_REALTIME_MODEL}",
          "policies": {
            "inbound": [
              {
                "module": "$import(./modules/realtime-policies)",
                "export": "protectInstructions"
              }
            ]
          }
        }
      },
      "policies": {
        "inbound": ["api-key-inbound", "set-openai-key"]
      }
    }
  }
}
```

The route-level `policies` run once, when the client opens the connection. The
`policies` inside `handler.options` run on every message. See
[Two different policies blocks](../handlers/websocket-pipeline-handler.mdx#configuration).

## 5. Connect a client

Connect to your gateway URL with a Zuplo API key instead of an OpenAI key. This
Node.js example uses the [`ws`](https://www.npmjs.com/package/ws) package:

```ts
import WebSocket from "ws";

const ws = new WebSocket("wss://my-gateway.zuplo.app/v1/realtime", {
  headers: {
    Authorization: `Bearer ${process.env.ZUPLO_API_KEY}`,
  },
});

ws.on("open", () => {
  console.log("Connected through Zuplo");
});

ws.on("message", (message) => {
  console.log(JSON.parse(message.toString()));
});

ws.on("close", (code, reason) => {
  console.log(`Closed: ${code} ${reason.toString()}`);
});
```

Replace `my-gateway.zuplo.app` with your gateway's hostname. A request without a
valid Zuplo API key gets a `401` response and no WebSocket connection. A client
that sends an event with `instructions`, such as `session.update` or
`response.create`, receives the error event and then a close with code `4003`.

## Next steps

- Add a [Rate Limiting](../policies/rate-limit-inbound.mdx) policy to the
  route-level `inbound` array to limit how many sessions each consumer opens.
- Run code when a session starts with the
  [`onOpen` connection hook](../handlers/websocket-pipeline-handler.mdx#the-onopen-connection-hook).
- Read more about message policies in the
  [WebSocket Pipeline Handler](../handlers/websocket-pipeline-handler.mdx)
  reference.