Using Custom AI Providers
Zuplo's AI Gateway supports the addition of custom AI providers. This allows users to route their AI Gateway Apps to self-hosted services and models, providing an additional layer of control, security and governance when using these models in production.
Requirements
A custom provider must expose an OpenAI-compatible API. The gateway expects:
POST <api-url>/v1/chat/completionsfor chat completion models, accepting and returning OpenAI Chat Completions schemasPOST <api-url>/v1/embeddingsfor embedding models, accepting and returning OpenAI Embeddings schemas
The gateway authenticates each request with your provider API key as a bearer
token: Authorization: Bearer <your-api-key>.
The gateway appends these paths to the API URL you configure, so the provider
must serve chat/completions under a /v1 path segment. A provider that serves
its OpenAI-compatible API at a path without /v1—directly off the origin, for
example—won't work as a custom provider.
Unsupported endpoints
Custom providers serve chat completions and embeddings only. The gateway's
/v1/responses and /v1/messages endpoints don't accept custom-provider
models: a /v1/responses request fails with a 400 error and the message
The /v1/responses endpoint is only supported by OpenAI provider. Current provider: <name>.
Adding a Custom AI Provider
To add a custom AI provider to your Zuplo AI Gateway, follow these steps:
-
Open Settings → AI Providers in your AI Gateway project in the Zuplo Portal.
-
Click on the Add Provider button.
-
Select the Custom option from the Custom Providers section of the list
-
Replace the pre-filled Provider Name (
custom) with a name for your provider. The name is how apps reference the provider's models: a custom provider namedacme-llmserves models asacme-llm/<model>. Names are lowercase (letters, numbers, dots, dashes, and underscores), and Zuplo reserves the built-in provider names (openai,anthropic,google,mistral,xai,moonshot,zuplo,zuplodemo,zuplo-demo). The name is permanent after creation. -
Enter the provider's API URL as its origin root—for example
https://api.together.xyz. The gateway appends/v1/chat/completions(and/v1/embeddingsfor embedding models) to this value itself, so the URL must not carry a/v1suffix.Leave the /v1 suffix off
Vendors usually publish a base URL that already ends in
/v1. Pasting that value makes the gateway call/v1/v1/chat/completions, and the provider answers404. Strip the suffix:Vendor Published base URL API URL to enter OpenRouter https://openrouter.ai/api/v1https://openrouter.ai/apiTogether https://api.together.xyz/v1https://api.together.xyzGroq https://api.groq.com/openai/v1https://api.groq.com/openaiFireworks https://api.fireworks.ai/inference/v1https://api.fireworks.ai/inference -
Enter the API Key for the selected provider (if there is no API key required, you can leave this blank).
-
Finally, add the available models that are hosted with your custom provider, along with their type. Optionally, you can add a dollar cost for input and output tokens if you wish to track this via the AI Gateway.
-
Click Create.
Verify the provider
Saving a provider triggers a production deployment of your gateway, and the
portal doesn't test the provider connection itself. Once the deployment
completes, send a chat completions request through your app's
Universal API URL with one of the provider's models and
confirm a 200 response. If the provider answers 404, check the API URL for a
stray /v1 suffix.
Modify, Update or Delete your Custom AI Provider
To modify, update, or delete an existing provider, open Settings → AI Providers and click the Edit or Delete icon next to the custom provider.
Further information can be found in the Managing Providers guide.