AI Gateway

AI Gateway Universal API

Applies to
Gatewayruntime for API, AI, and MCP gateways
Hosting
Managed EdgeManaged DedicatedSelf-Hosted

Zuplo AI Gateway provides a universal API that standardizes interactions with various AI providers. This API follows the OpenAI API specification, making it easy to integrate with existing apps that already use OpenAI's API.

Using the Universal API

Every AI Gateway app serves the Universal API. The app page in the portal shows each app's API URL, which ends in the app's ID—for example https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e. Client libraries expect a base URL ending in /v1, so append that suffix. When the gateway runs the authentication policy, send the app's API key as a bearer token.

Models are referenced as providerName/model, where providerName is the provider name configured in your gateway. This is how the gateway knows which provider to route each request to—and it means one app can use models from several providers through the same endpoint.

If you are using an SDK or library that supports custom base URLs, you can configure it to use your app's URL. For example, with the OpenAI Node.js SDK, you can set the baseURL option:

TypeScriptCode
import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.ZUPLO_APP_API_KEY, baseURL: "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1", }); const response = await client.chat.completions.create({ model: "openai/gpt-6-luna", messages: [ { role: "user", content: "Write a one-sentence bedtime story about a unicorn.", }, ], }); console.log(response.choices[0].message.content);

If the app's Model Filtering policy has an allow list, a request that omits model uses the first model in the list—so clients that can't set a model still work.

Supported endpoints

EndpointNotes
/v1/chat/completionsChat completions for every provider
/v1/embeddingsEmbeddings for the providers marked in the capability matrix
/v1/responsesOpenAI Responses API: OpenAI, xAI, and the OpenAI-compatible models that serve it on Azure AI, Bedrock Mantle and OpenRouter
/v1/messagesAnthropic Messages API: Anthropic, and the Claude models of Azure AI, Bedrock Mantle, Vertex AI and OpenRouter
/v1/messages/count_tokensCounts the tokens of a Messages request without running it: Anthropic, Vertex AI, and the Claude models of Bedrock Mantle. Other providers return a 400
GET /v1/responses/{id}Retrieves a stored response from a provider that serves the Responses API
DELETE /v1/responses/{id}Deletes a stored response
GET /v1/modelsLists the models the app can call. See Listing models

Jev is the exception: it serves TypeSafe's System One API on /v1/systemone, which takes a state and typed questions rather than chat messages. That endpoint isn't part of the Universal API. See Jev.

Stored responses

Retrieving or deleting a stored response sends no body, so the gateway has no model to route on. It uses the app's first configured completions model, which has to be on a provider that serves the Responses API. A GET /v1/responses/{id}/input_items request, which lists the input items of a stored response, works the same way. For the setup each provider needs, see Azure AI and Bedrock Mantle.

Listing models

GET /v1/models lists the models your app can call, as qualified IDs such as openai/gpt-5. Send GET /v1/models/{model} to look up one of them. The list holds only active models of providers that have credentials, that serve the endpoint you ask about, and that the app's Model Filtering policy allows.

A model list is per endpoint, because each endpoint serves different models. The gateway picks the endpoint and the response format from the request:

  • With an anthropic-version header, as the Anthropic SDK sends, the list holds the models that serve /v1/messages, in the Messages format. It pages with limit (1 to 1000, default 20) and one of after_id or before_id. The response carries first_id, last_id, and has_more.
  • Without it, the list holds the models that serve /v1/chat/completions, in the OpenAI format, with no paging.

To list the models of another endpoint, set the x-zuplo-models-endpoint header to chat-completions, messages, responses, embeddings, or systemone. Any other value returns a 400, and any method other than GET returns a 405.

TerminalCode
curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/models \ -H "Authorization: Bearer $ZUPLO_APP_API_KEY" \ -H "x-zuplo-models-endpoint: embeddings"

Request headers

The gateway forwards your request headers to the provider. A header your client or your own policies set—a session ID, a trace header, a feature flag—arrives with the request.

The gateway removes what it owns first:

RemovedWhy
authorization, x-api-key, api-key, proxy-authorization, and cookiesCredentials addressed to the gateway. The provider gets its own credential from your provider configuration instead.
content-type, accept, anthropic-version, anthropic-betaThe gateway controls the wire format it sends upstream, which isn't always the one your client asked for.
openai-organization, openai-project, x-goog-api-key, x-goog-user-projectAnother vendor's credentials and tenant selectors. They cause a 401 as soon as a request routes to a provider they don't belong to.
host, connection, transfer-encoding, content-length, and the rest of the hop-by-hop setThey describe the connection to the gateway, not the one to the provider.
cdn-loop, and anything starting with zp-, cf-, or x-amz-They describe the path your request took to reach the gateway.

Everything else reaches the provider, including user-agent, x-forwarded-for, true-client-ip, idempotency-key, x-request-id, and the x-stainless-* headers the OpenAI and Anthropic SDKs set.

To change what the provider receives, add a header policy—set-headers-inbound, remove-headers-inbound, or clear-headers-inbound—to the app's policy chain. Headers are copied after the chain runs, so a policy that adds one puts it on the provider request too. Setting Authorization, x-api-key, or api-key in a policy doesn't override the credential from your provider configuration.

Bedrock Runtime isn't part of the Universal API and forwards a different set, described on its own page.

Last modified on