# Using Vertex AI

**Vertex AI** is Google Cloud's managed model platform. Adding it as a provider
serves both the Gemini family and Vertex's Model Garden partner models—DeepSeek,
Qwen, GLM, Kimi, MiniMax, Gemma, and GPT-OSS—to your [apps](./apps.mdx) through
the [Universal API](./universal-api.mdx), running on your Google Cloud project
and your Google Cloud billing.

Chat completions go to Vertex's **OpenAI-compatible endpoint**, so the gateway
forwards OpenAI-shaped requests without translating them. Embeddings work too,
though Vertex has no OpenAI-compatible embeddings endpoint—the gateway
translates those onto Vertex's own embedding APIs, which is invisible from your
app. What sets this provider apart is how you connect it: the endpoint is
per-location, the Google Cloud project is a separate setting, and the credential
is a **service account key file**—not an API key.

:::caution{title="Vertex AI doesn't accept API keys"}

Every other AI Gateway provider takes an API key. Vertex doesn't: its prediction
endpoints reject API keys outright, answering that they expect an OAuth2 access
token instead. The provider dialog asks for the JSON key file of a Google Cloud
service account, and the gateway exchanges it for short-lived access tokens on
your behalf. Pasting an API key into the field fails validation before the
provider is saved.

:::

## Model references include the publisher prefix

Apps reference a provider's models as `providerName/model`. Every Vertex model
ID is publisher-qualified, so a Vertex model reference has two slashes in it:

```
vertexai/google/gemini-3.7-flash
└──┬───┘ └──────────┬──────────┘
provider          model
```

Everything before the **first** slash is the provider name you chose, and
everything after it is the model ID exactly as Vertex expects it. The publisher
isn't always `google`—Model Garden models carry their own vendor's prefix. A
provider named `vertexai` serves all of these:

- `vertexai/google/gemini-3.7-flash`
- `vertexai/qwen/qwen3-coder-480b-a35b-instruct-maas`
- `vertexai/deepseek-ai/deepseek-v3.2-maas`
- `vertexai/zai-org/glm-5-maas`

Copy model IDs from the model picker rather than typing them—the `-maas` suffix
and the publisher prefix are both part of the ID.

## How the gateway calls Vertex AI

When no valid cached token exists, the gateway uses the service account key to
request a short-lived Google OAuth 2.0 access token. It caches and refreshes
that token, then calls Vertex's OpenAI-compatible endpoint with the token as the
bearer credential.

<Diagram height="h-64">
  <DiagramNode id="app">Your app</DiagramNode>
  <DiagramNode id="gateway" variant="zuplo">
    AI Gateway
  </DiagramNode>
  <DiagramNode id="oauth" variant="orange">
    Google OAuth
  </DiagramNode>
  <DiagramGroup id="gcp" label="Your Google Cloud project">
    <DiagramNode id="compat" variant="blue">
      OpenAI-compatible endpoint
    </DiagramNode>
    <DiagramNode id="embed" variant="green">
      Vertex embedding APIs
    </DiagramNode>
  </DiagramGroup>
  <DiagramEdge from="app" to="gateway" label="Universal API" />
  <DiagramEdge
    from="gateway"
    to="oauth"
    label="service account key → access token"
    lineStyle="dashed"
  />
  <DiagramEdge from="gateway" to="compat" label="chat completions" />
  <DiagramEdge from="gateway" to="embed" label="embeddings" />
</Diagram>

The request URL the gateway builds combines both settings you enter:

```
https://us-central1-aiplatform.googleapis.com
  /v1/projects/my-project-123/locations/us-central1/endpoints/openapi/chat/completions
```

The **location** lives in the hostname, which is why the provider dialog asks
for a location instead of an endpoint URL. The **project** doesn't appear in the
hostname at all—it appears only in the path—so it can't be derived from the
endpoint and is a field of its own. That's why the project is required rather
than optional: without it the gateway would have to guess, and a guess would
send your traffic and your spend to someone else's Google Cloud project.

## Supported endpoints

| Endpoint               | Vertex AI models |
| ---------------------- | ---------------- |
| `/v1/chat/completions` | ✅ Forwarded     |
| `/v1/embeddings`       | ✅ Translated    |
| `/v1/responses`        | ❌               |
| `/v1/messages`         | ❌               |

Streaming (`stream: true`) works on chat completions.

The gateway rejects `/v1/responses` and `/v1/messages` requests for Vertex
models with a `400` error before any upstream call, because Vertex serves
neither the OpenAI Responses API nor the Anthropic Messages API.

:::note{title="Anthropic models on Vertex aren't supported"}

Vertex also hosts Anthropic's Claude models, but on a different API surface than
the one this provider uses. The gateway can't route to them. To use Claude
models, add the [Anthropic](./providers.mdx) provider or
[Bedrock Mantle](./bedrock-mantle.mdx).

:::

## Model availability depends on the location

Google registers each Vertex model in specific locations, and it changes that
list over time. Most Model Garden models are served only from the **global**
location; the Gemini models are more widely available.

The model picker doesn't filter by your provider's location, so every model
stays selectable from every provider. If you call a model that isn't served
where your provider points, Vertex answers with a `400` that names both:

```
Publisher model 'publishers/qwen/models/qwen3-coder-480b-a35b-instruct-maas'
is not available in region 'us-central1'
```

Set the provider's **Location** to `global` if you plan to use Model Garden
models, or add a second Vertex provider for a specific region and route to
whichever one serves the model you want.

## Before you begin

You need:

- A Google Cloud project with the **Vertex AI API**
  (`aiplatform.googleapis.com`) enabled and billing enabled. Gemini models
  require both.
- A service account in that project holding the **Vertex AI User**
  (`roles/aiplatform.user`) role, and a **JSON key** for it. The steps are in
  [Set up Google Cloud](#set-up-google-cloud).
- The project's **project ID**—the readable identifier such as `my-project-123`,
  not the numeric project number.
- An AI Gateway project in the Zuplo Portal.
- An AI Gateway [app](./apps.mdx) to call the models from. The app page shows
  the app's API URL, and its API key lives on the app's **API Key** tab.

## Set up Google Cloud

Do this before adding the provider—the key file you download in the last step is
what the provider dialog asks for.

<Stepper>

1. In the [Google Cloud console](https://console.cloud.google.com/), select the
   project you want to serve models from and note its **project ID**. The
   console shows the ID next to the project name, alongside the project number;
   the gateway needs the ID.

1. Enable the **Vertex AI API** on the project, under **APIs & Services →
   Library**. Confirm billing is enabled for the project as well.

1. Create a service account under **IAM & Admin → Service Accounts**.

1. Grant that service account the **Vertex AI User** (`roles/aiplatform.user`)
   role **on the same project**. This is the smallest role that can call
   prediction endpoints. Vertex AI Administrator grants more than the gateway
   needs, and Vertex AI Viewer can't make predictions at all.

1. Create a JSON key for the service account: open the service account, go to
   **Keys → Add key → Create new key**, choose **JSON**, and download the file.
   Keep it somewhere safe—Google doesn't let you download it again.

</Stepper>

:::note

Enabling the API and granting the role are separate steps, and missing the first
one is the most common setup failure. It surfaces as a `403` that talks about
the Vertex AI API rather than about permissions, which reads like a bad key.

:::

## Add the provider

Adding or editing providers requires the **Edit** permission, granted to Zuplo
account and project **Admins**—see
[Managing Providers](./managing-providers.mdx).

<Stepper>

1. Open
   [**Settings → AI Providers**](https://portal.zuplo.com/+/account/project/ai/settings/data-models)
   in your AI Gateway project in the Zuplo Portal.

1. Click the **Add Provider** button.

1. In the **AI Provider** list, select **Vertex AI** from the Default Providers
   group.

1. Review the **Provider Name**, which fills in as `vertexai` (a second
   configuration becomes `vertexai-2`). You can replace it with your own name,
   but only now—the name is permanent after creation, and it's the prefix in
   every model reference: a provider named `vertexai` serves
   `vertexai/google/gemini-3.7-flash`.

1. In **Location**, enter the Vertex AI location to call, such as `us-central1`
   or `europe-west4`. Enter `global` to use Vertex's global endpoint. The
   gateway derives the endpoint from this value—there's no URL to enter.

1. In **Google Cloud Project ID**, enter the project ID you noted earlier, such
   as `my-project-123`. The dialog rejects an all-digits value, which is the
   project number rather than the ID.

1. In **Service Account Key (JSON)**, paste the entire contents of the JSON key
   file you downloaded. Paste it whole—a truncated copy fails validation.

1. Select the models to enable, or click **Select All**. The picker shows
   Google's per-token prices, which the gateway uses to track cost per app. You
   can change the selection later.

1. Click **Create**.

</Stepper>

:::note

Saving provider settings triggers an automatic production deployment of your
gateway, because provider credentials are part of the deployed gateway. The
change is live once the deployment completes.

:::

:::note{title="Prices in the picker are a snapshot"}

The per-token prices shown in the model picker are recorded rates, not a live
read of Google's price list, and some Gemini models are on promotional rates
that Google marks as temporary. Treat the gateway's cost figures as an estimate
and your Google Cloud bill as the record.

:::

When you edit the provider later—see
[Managing Providers](./managing-providers.mdx)—the **Location** and **Google
Cloud Project ID** fields show the saved values and stay editable, and you can
replace the key file. Leave the key field blank to keep the stored key. The
**Provider Name** doesn't change.

## Verify the provider

Once the deployment completes, send a chat completions request to your app's
[Universal API](./universal-api.mdx) URL—shown at the top of the
[app page](./apps.mdx)—with the app's API key as the bearer token and one of the
models you enabled:

```bash
curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions \
  -H "Authorization: Bearer $ZUPLO_APP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertexai/google/gemini-3.7-flash",
    "messages": [{ "role": "user", "content": "Say hi" }]
  }'
```

The URL is a sample—replace it with your app's API URL plus
`/v1/chat/completions`, set `ZUPLO_APP_API_KEY` to the app's API key, and
substitute any model you enabled for `vertexai/google/gemini-3.7-flash`.

A `200` response confirms the provider works; the body carries
`"provider": "vertex-ai"`—the provider type, not the name you chose. If the
request fails immediately after you save the provider, the deployment may not
have finished—retry before debugging further.

## Call the models

Use any OpenAI client with your app's URL plus `/v1` as the base URL and the
app's API key:

```ts
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ZUPLO_APP_API_KEY,
  baseURL:
    "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
});

const response = await client.chat.completions.create({
  model: "vertexai/google/gemini-3.7-flash",
  messages: [{ role: "user", content: "Summarize this ticket." }],
});
```

Streaming works the same way—set `stream: true` and read the
`chat.completion.chunk` events.

### Embeddings

Embeddings use the same client and the same `/v1/embeddings` endpoint as any
other provider:

```ts
const embeddings = await client.embeddings.create({
  model: "vertexai/google/gemini-embedding-2",
  input: ["first document", "second document"],
});
```

`dimensions` and `encoding_format` both work. Two Vertex-specific limits apply,
and the gateway enforces both before calling Vertex, so you get a `400` rather
than a partial charge:

- **At most 250 inputs per request.** Split larger batches yourself.
- **Text input only.** Pre-tokenized input—arrays of token IDs, which the OpenAI
  API also accepts—is rejected, because Vertex has no way to represent it. Send
  the original text.

## Troubleshooting

**The dialog rejects your credential.** Vertex uses a service account, not an
API key. The field takes the whole JSON key file downloaded from **IAM & Admin →
Service Accounts → Keys**. The dialog also rejects a workload identity
federation configuration—its `type` is `external_account` rather than
`service_account`—and a file whose `private_key` isn't a PEM-encoded key. Paste
the file exactly as downloaded, keeping the `\n` escapes intact.

**The dialog rejects your project ID.** Project IDs are 6 to 30 characters:
lowercase letters, digits, and hyphens, starting with a letter and not ending in
one. An all-digits value is the project number, which the Google Cloud console
shows next to the ID; the gateway needs the ID.

**A request fails with a `403` about the Vertex AI API.** The API isn't enabled
on the project. Enable **Vertex AI API** under **APIs & Services → Library** and
confirm the project has billing enabled. This error names the API rather than
permissions, so it's easy to mistake for a credential problem.

**A request fails with a permission error.** The service account is missing the
**Vertex AI User** (`roles/aiplatform.user`) role, or holds it on a different
project than the one configured on the provider. Grant the role on the same
project you entered as the **Google Cloud Project ID**.

**A request fails with `400` saying the model isn't available in the region.**
The model isn't served from your provider's location. Most Model Garden models
are global-only—see
[Model availability](#model-availability-depends-on-the-location). Set the
provider's **Location** to `global`, or use a provider that points at a location
serving that model.

**An embeddings request fails with `400` about the input.** The gateway accepts
at most 250 inputs per request and text input only. See
[Embeddings](#embeddings).

**A `/v1/responses` or `/v1/messages` request fails with `400`.** Vertex's
OpenAI-compatible endpoint serves neither API. Use `/v1/chat/completions`, or
another provider for those endpoints—see
[Supported endpoints](#supported-endpoints).

## Next steps

- [AI Providers](./providers.mdx)—the capability matrix across every supported
  provider.
- [Universal API](./universal-api.mdx)—the endpoints every app serves and how
  model references work.
- [Managing Providers](./managing-providers.mdx)—edit models, keys, the
  location, and the project, and understand when changes deploy.
- [AI Gateway Apps](./apps.mdx)—create the apps that call your Vertex-backed
  models.
- [Model Filtering policy](../policies/ai-gateway-model-filtering-v2-inbound.mdx)—control
  which models each app can call.