AI Gateway

Zuplo AI Gateway Getting Started

This guide walks you through setting up your first AI Gateway project, from connecting a Git repository to making your first LLM request through Zuplo.

Prerequisites

A Zuplo account (sign up free at zuplo.com)
An account with a supported Git provider that can create repositories (this guide uses GitHub)
An API key for at least one LLM provider (OpenAI, Anthropic, Google, Mistral, xAI, etc.)—or none at all, if you start with the free Zuplo Demo provider

Create an AI Gateway project

Log into the Zuplo Portal, navigate to Projects, and click New Project. Choose Configure an AI gateway, give your project a name (for example, "MyCompany AI Gateway"), and click Create Project.
Connect the project to a Git repository

Your AI Gateway deploys from source control, so the first step after creating the project is connecting it to a repository. The project shows a Let's Get You Deployed screen until a repository is connected.

Zuplo supports GitHub, GitLab, Bitbucket, and Azure DevOps. This guide uses GitHub, which is the only provider with automatic deployments—see Source Control and Deployment for the other providers, which deploy through the Zuplo CLI in your own CI/CD pipeline.

With GitHub, click Create New Repo to open GitHub with the repository name prefilled, create the repository, then return to the Zuplo Portal and select the repository to connect it. Zuplo adds the gateway's source to the repository and deploys it. After that, pushes to your default branch deploy to production.

To work with the gateway's source later—for example, to add custom policies—clone the repository:
```
Code
git clone https://github.com/your-org/your-gateway-repo.git
```
If your default branch requires pull requests, Zuplo pushes the gateway source to a setup branch and shows you a pull request to merge. The project finishes connecting once the pull request lands on the default branch.

See Source Control for what the repository contains and how deployments work.
Configure a provider

Providers are the LLM services (like OpenAI or Anthropic) that your apps use. You configure these once as an administrator, and your team members use them without needing direct access to provider API keys.

Open Settings → AI Providers and click Add Provider. Select your AI provider (for example, OpenAI), paste your provider's API key, and select which models to make available. The Provider Name field fills in automatically based on the provider you selected—openai, for example. Click Create.

The provider name becomes the model prefix
Apps reference this provider's models as providerName/model, so the provider name is the prefix in every model string—a provider named openai serves openai/gpt-5-mini. The pre-filled name is usually what you want. To use your own, set it now—the name is permanent after creation—and keep it short and lowercase.

Repeat for additional providers. See AI Providers for the full list of supported providers, including OpenAI-compatible custom providers.
Create a team

Teams organize users and carry hierarchical budget controls and policy templates. Even if you're starting solo, you need at least one team.

Open the Teams tab and click Create Team. Name your team (for example, "Root" or your company name), choose an icon, and click Create Team. To set organization-wide spending limits, open the gateway's Settings → Usage Limits; for team-specific limits, use the team's Usage & Limits tab. See Usage Limits.
Create an app

Apps represent individual projects or services that call the AI Gateway. Each app gets its own unique URL and API key.

Open the Apps tab and click Create App. Give the app a descriptive name (for example, "Tennis Chat") and select the team that owns it. That's the whole dialog—models, budgets, and other behavior are configured on the app's policy chain in the next step.

If the app's team has a policy template, the new app starts with the template's policy chain already in place. See Policy Templates.
Review the app's policy chain

The portal opens the app's Policies tab, which shows the app's policy chain—the ordered list of policies that run on each request. No policy is required: an app with an empty chain works immediately, and each request picks its own model as providerName/model, from any model your providers expose.

To restrict that, add the Model Filtering policy and give it an allow list such as openai/gpt-5-mini. Requests may then use only listed models, and the first entry becomes the default when a request omits model.

An unrestricted app can reach every model you configured
Without Model Filtering, an app may call any model available through the providers assigned to its team, including expensive ones. Add Model Filtering for any app that should be limited to a specific set.

This is also where you add budgets, semantic caching, guardrails, and any custom policies. Changes to the chain apply within about a minute—no redeploy needed. See Policy Chains.
Get the app's URL and API key

The app page shows its API URL and API Key at the top right. The API URL is the gateway's hostname plus the app's ID—for example https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e. Copy both rather than constructing them.

Client libraries expect a base URL ending in /v1, so append it: the OpenAI SDK examples below use https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1 as a stand-in, and requests land on paths like /v1/chat/completions.

The gateway checks the key only after you add the authentication policy to the app's chain. Until then the gateway identifies the app from the {app_id} segment of the URL and accepts the request without a key. The examples below send the key so they keep working once you add the policy.

Export the key so the samples below can read it:
```
Code
export ZUPLO_APP_API_KEY="<the API key from the app page>"
```
Send your first request from the Playground

The app's Playground tab chats with the app's own gateway URL and API key, so it exercises the same policy chain your code will hit—without wiring up a client first.

Pick a model from the selector, optionally set a system prompt, and send a message. The tab shows the cURL for the next request, and Copy cURL puts it on your clipboard to run in a terminal or adapt it for your client. Expand any reply to inspect the request and the raw response.

Integrate with your own code

Once the Playground works, configure your own code with the same URL and key.

These examples use the official Node.js OpenAI SDK; run npm install openai first. For other clients, see the integration guides for the OpenAI SDK, AI SDK, and LangChain.

Before: direct provider integration (sample.mjs)


Code
import OpenAI from "openai";

// Old approach - directly calling OpenAI
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

const completion = await openai.chat.completions.create({
  model: "gpt-5-mini",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(completion.choices[0].message.content);

After: using the Zuplo AI Gateway (sample.mjs)


Code
import OpenAI from "openai";

// New approach - using Zuplo AI Gateway
const openai = new OpenAI({
  apiKey: process.env.ZUPLO_APP_API_KEY,
  baseURL:
    "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
});

const completion = await openai.chat.completions.create({
  model: "openai/gpt-5-mini",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(completion.choices[0].message.content);

Run the example with node sample.mjs.

What changed?

URL: Replace your provider's URL with your app's gateway URL from the app page
API Key: Use your app's API key instead of the provider's key—the gateway checks it once the authentication policy applies
Model: Reference models as providerName/model so the gateway knows which provider to route to
Everything else stays the same: The request format remains compatible with OpenAI's API

Verify your setup

Make your first request

Send a test request through your gateway, using your app's URL and API key:

Code
curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions \
  -H "Authorization: Bearer YOUR_APP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5-mini",
    "messages": [{"role": "user", "content": "Hello, world!"}]
  }'

Check your dashboard

Open the Apps tab of your AI Gateway project
Click on your app
Click on the app's Dashboard tab to view:
- Request count
- Token usage
- Time to first byte
- Current spending

You should see your test request appear with token usage and performance metrics.

Next steps

Policy Chains: add budgets, semantic caching, and guardrails to your app's chain
Custom Policies: write your own policy and add it to an app's chain
Policy Templates: give every new app in a team a consistent starting chain
Usage Limits: set spending limits for the gateway, teams, and apps

Common issues

Symptom	Fix
"Authentication failed" error	Verify you're using your app's API key, not your provider's key. This error appears only when the authentication policy applies
404 "Unsupported AI Gateway endpoint"	The path must end with a supported `/v1/...` operation—usually the URL is missing its `/v1` segment. Use the app's API URL plus `/v1/chat/completions`, for example
400 error asking for `providerName/model`	Prefix the model with the provider name configured in the portal—`openai/gpt-5-mini`, not `gpt-5-mini`
403 error listing the allowed models	The app's Model Filtering policy doesn't include the requested model; add it on the app's Policies tab or use a listed one
Budget limit reached immediately	Check the limits at every level—gateway, team, and the app's Budgets and Costs policy—since any level's limit blocks the request

Edit this page

Last modified on August 7, 2026

Introduction Source Control