Zuplo
Providers

Using Custom AI Providers

Zuplo's AI Gateway supports the addition of custom AI providers. This allows users to route their AI Gateway Apps to self-hosted services and models, providing an additional layer of control, security and governance when using these models in production.

Requirements

A custom provider must expose an OpenAI-compatible API. The gateway expects:

  • POST <api-url>/v1/chat/completions for chat completion models, accepting and returning OpenAI Chat Completions schemas
  • POST <api-url>/v1/embeddings for embedding models, accepting and returning OpenAI Embeddings schemas

The gateway authenticates each request with your provider API key as a bearer token: Authorization: Bearer <your-api-key>.

The gateway appends these paths to the API URL you configure, so the provider must serve chat/completions under a /v1 path segment. A provider that serves its OpenAI-compatible API at a path without /v1—directly off the origin, for example—won't work as a custom provider.

Unsupported endpoints

Custom providers serve chat completions and embeddings only. The gateway's /v1/responses and /v1/messages endpoints don't accept custom-provider models: a /v1/responses request fails with a 400 error and the message The /v1/responses endpoint is not supported by provider '<name>'.

Adding a Custom AI Provider

To add a custom AI provider to your Zuplo AI Gateway, follow these steps:

  1. Open Settings → AI Providers in your AI Gateway project in the Zuplo Portal.

  2. Click on the Add Provider button.

  3. Select the Custom option from the Custom Providers section of the list

  4. Replace the pre-filled Provider Name (custom) with a name for your provider. The name is how apps reference the provider's models: a custom provider named acme-llm serves models as acme-llm/<model>. Names are lowercase (letters, numbers, dots, dashes, and underscores), and Zuplo reserves the built-in provider names (openai, anthropic, google, mistral, xai, moonshot, zuplo, zuplodemo, zuplo-demo, bedrockmantle, bedrock-mantle, azureai, azure-ai, vertexai, vertex-ai, openrouter, open-router, jev). The name is permanent after creation.

  5. Enter the provider's API URL as its origin root—for example https://api.together.xyz. The gateway appends /v1/chat/completions (and /v1/embeddings for embedding models) to this value itself, so the URL must not carry a /v1 suffix.

    Leave the /v1 suffix off

    Vendors usually publish a base URL that already ends in /v1. Pasting that value makes the gateway call /v1/v1/chat/completions, and the provider answers 404. Strip the suffix:

    VendorPublished base URLAPI URL to enter
    OpenRouterhttps://openrouter.ai/api/v1https://openrouter.ai/api
    Togetherhttps://api.together.xyz/v1https://api.together.xyz
    Groqhttps://api.groq.com/openai/v1https://api.groq.com/openai
    Fireworkshttps://api.fireworks.ai/inference/v1https://api.fireworks.ai/inference

    OpenRouter is built in

    OpenRouter is now a built-in provider—pick it from the provider list instead of configuring it here, and you get model discovery and per-model cost tracking. See Using OpenRouter. The OpenRouter row above still applies to any OpenRouter provider you added before it became built-in.

  6. Enter the API Key for the selected provider (if there is no API key required, you can leave this blank).

  7. Finally, add the available models that are hosted with your custom provider, along with their type. Optionally, you can add a dollar cost for input and output tokens if you wish to track this via the AI Gateway.

  8. Click Create.

Verify the provider

Saving a provider triggers a production deployment of your gateway, and the portal doesn't test the provider connection itself. Once the deployment completes, send a chat completions request through your app's Universal API URL with one of the provider's models and confirm a 200 response. If the provider answers 404, check the API URL for a stray /v1 suffix.

Modify, Update or Delete your Custom AI Provider

To modify, update, or delete an existing provider, open Settings → AI Providers and click the Edit or Delete icon next to the custom provider.

Further information can be found in the Managing Providers guide.

Last modified on
JevOverview