Using Vertex AI

Vertex AI is Google Cloud's managed model platform. Adding it as a provider serves the Gemini family, Vertex's Model Garden partner models—DeepSeek, Qwen, GLM, Kimi, MiniMax, Gemma, and GPT-OSS—and Anthropic's Claude models to your apps through the Universal API, running on your Google Cloud project and your Google Cloud billing.

Chat completions go to Vertex's OpenAI-compatible endpoint, so the gateway forwards OpenAI-shaped requests without translating them. Embeddings work too, though Vertex has no OpenAI-compatible embeddings endpoint—the gateway translates those onto Vertex's own embedding APIs, which is invisible from your app. Claude models are different again: Vertex serves them on the native Anthropic Messages API, so they're available on /v1/messages only—see Call Claude models on the Messages API. What sets this provider apart is how you connect it: the endpoint is per-location, the Google Cloud project is a separate setting, and the credential is a service account key file—not an API key.

Model references include the publisher prefix

Apps reference a provider's models as providerName/model. Every Vertex model ID is publisher-qualified, so a Vertex model reference has two slashes in it:

Code
vertexai/google/gemini-3.7-flash
└──┬───┘ └──────────┬──────────┘
provider          model

Everything before the first slash is the provider name you chose, and everything after it is the model ID exactly as Vertex expects it. The publisher isn't always google—Model Garden models carry their own vendor's prefix. A provider named vertexai serves all of these:

• vertexai/google/gemini-3.7-flash
• vertexai/qwen/qwen3-coder-480b-a35b-instruct-maas
• vertexai/deepseek-ai/deepseek-v3.2-maas
• vertexai/zai-org/glm-5-maas
• vertexai/anthropic/claude-haiku-4-5@20251001

Copy model IDs from the model picker rather than typing them—the -maas suffix, the @ version on the dated Claude models, and the publisher prefix are all part of the ID.

How the gateway calls Vertex AI

When no valid cached token exists, the gateway uses the service account key to request a short-lived Google OAuth 2.0 access token. It caches and refreshes that token, then calls Vertex's OpenAI-compatible endpoint with the token as the bearer credential.

The request URL the gateway builds combines both settings you enter:

Code
https://us-central1-aiplatform.googleapis.com
  /v1/projects/my-project-123/locations/us-central1/endpoints/openapi/chat/completions

For a Claude model the gateway calls Vertex's Anthropic publisher surface instead, with the model in the URL:

Code
https://aiplatform.googleapis.com
  /v1/projects/my-project-123/locations/global/publishers/anthropic/models/claude-haiku-4-5@20251001:rawPredict

The location lives in the hostname, which is why the provider dialog asks for a location instead of an endpoint URL. The project doesn't appear in the hostname at all—it appears only in the path—so it can't be derived from the endpoint and is a field of its own. That's why the project is required rather than optional: without it the gateway would have to guess, and a guess would send your traffic and your spend to someone else's Google Cloud project.

Supported endpoints

Forwarded means the gateway sends your request on in the shape you wrote it. Translated means it converts between two API shapes—for embeddings, from the OpenAI shape onto Vertex's own embedding APIs, which is invisible from your side.

Streaming (stream: true) works on chat completions and on Messages.

The gateway's model catalog records which API serves each model, and the gateway routes every request accordingly. It rejects a request for a model on an endpoint that doesn't serve it with a 400 before any upstream call: /v1/responses for every Vertex model, /v1/messages for Gemini and Model Garden models, and /v1/chat/completions or /v1/embeddings for Claude models—that last error names /v1/messages as the endpoint to use.

Model availability depends on the location

Google registers each Vertex model in specific locations, and it changes that list over time. Most Model Garden models are served only from the global location; the Gemini models are more widely available.

The model picker doesn't filter by your provider's location, so every model stays selectable from every provider. If you call a model that isn't served where your provider points, Vertex answers with a 400 that names both:

Code
Publisher model 'publishers/qwen/models/qwen3-coder-480b-a35b-instruct-maas'
is not available in region 'us-central1'

Set the provider's Location to global if you plan to use Model Garden models or Claude models. Google's regional endpoints serve Claude Sonnet 4.6 and earlier, only in the regions listed on each model's Model Garden card, and only where the model is enabled for your project. Claude models newer than Sonnet 4.6 are served by the global endpoint (and by Google's us and eu multi-region endpoints, which the gateway doesn't accept as a location yet). For a Claude model in a location that doesn't serve it, or where it isn't enabled for your project, Vertex answers a 400 saying the model "is not servable in region". Alternatively, add a second Vertex provider for a specific region and route to whichever one serves the model you want.

Before you begin

You need:

• A Google Cloud project with the Vertex AI API (aiplatform.googleapis.com) enabled and billing enabled. Gemini models require both.
• A service account in that project holding the Vertex AI User (roles/aiplatform.user) role, and a JSON key for it. The steps are in Set up Google Cloud.
• The project's project ID—the readable identifier such as my-project-123, not the numeric project number.
• An AI Gateway project in the Zuplo Portal.
• An AI Gateway app to call the models from. The app page shows the app's API URL, and its API key lives on the app's API Key tab.

Set up Google Cloud

Do this before adding the provider—the key file you download in the last step is what the provider dialog asks for.

Add the provider

Adding or editing providers requires the Edit permission, granted to Zuplo account and project Admins—see Managing Providers.

When you edit the provider later—see Managing Providers—the Location and Google Cloud Project ID fields show the saved values and stay editable, and you can replace the key file. Leave the key field blank to keep the stored key. The Provider Name doesn't change.

Verify the provider

Once the deployment completes, send a chat completions request to your app's Universal API URL—shown at the top of the app page—with the app's API key as the bearer token and one of the models you enabled:

Code
curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions \
  -H "Authorization: Bearer $ZUPLO_APP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertexai/google/gemini-3.7-flash",
    "messages": [{ "role": "user", "content": "Say hi" }]
  }'

The URL is a sample—replace it with your app's API URL plus /v1/chat/completions, set ZUPLO_APP_API_KEY to the app's API key, and substitute any model you enabled for vertexai/google/gemini-3.7-flash.

A 200 response confirms the provider works; the body carries "provider": "vertex-ai"—the provider type, not the name you chose. If the request fails immediately after you save the provider, the deployment may not have finished—retry before debugging further.

Call the models

Use any OpenAI client with your app's URL plus /v1 as the base URL and the app's API key:

Code
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ZUPLO_APP_API_KEY,
  baseURL:
    "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
});

const response = await client.chat.completions.create({
  model: "vertexai/google/gemini-3.7-flash",
  messages: [{ role: "user", content: "Summarize this ticket." }],
});

Streaming works the same way—set stream: true and read the chat.completion.chunk events.

For the Vercel AI SDK, use @ai-sdk/openai-compatible for Gemini and Model Garden models and @ai-sdk/anthropic for Claude—see the Vertex AI section of the AI SDK page. The @ai-sdk/google-vertex provider doesn't work; see Troubleshooting.

Embeddings

Embeddings use the same client and the same /v1/embeddings endpoint as any other provider:

Code
const embeddings = await client.embeddings.create({
  model: "vertexai/google/gemini-embedding-2",
  input: ["first document", "second document"],
});

dimensions and encoding_format both work. Two Vertex-specific limits apply, and the gateway enforces both before calling Vertex, so you get a 400 rather than a partial charge:

• At most 250 inputs per request. Split larger batches yourself.
• Text input only. Pre-tokenized input—arrays of token IDs, which the OpenAI API also accepts—is rejected, because Vertex has no way to represent it. Send the original text.

Call Claude models on the Messages API

Claude models serve the native Anthropic Messages API at /v1/messages, including streaming and /v1/messages/count_tokens. With the Anthropic SDK, set baseURL to the app's URL without /v1—the SDK appends /v1/messages itself—and pass the app's API key as authToken, not apiKey:

Code
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  baseURL:
    "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e",
  authToken: process.env.ZUPLO_APP_API_KEY, // apiKey would send x-api-key, which the gateway ignores
});

const message = await client.messages.create({
  model: "vertexai/anthropic/claude-haiku-4-5@20251001",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Say hi" }],
});

The gateway takes care of what Vertex does differently from Anthropic's own API: it addresses the model in the request URL and drops it from the body, adds the anthropic_version field Vertex requires, and authenticates with the access token it minted from your service account. Your request body is otherwise forwarded as-is, so system prompts, tool use, and streaming work as they do against Anthropic directly. Client headers such as anthropic-beta pass through, narrowed to the values Vertex accepts.

Claude models aren't available on /v1/chat/completions—a request there returns a 400 that points at /v1/messages.

Troubleshooting

The dialog rejects your credential. Vertex uses a service account, not an API key. The field takes the whole JSON key file downloaded from IAM & Admin → Service Accounts → Keys. The dialog also rejects a workload identity federation configuration—its type is external_account rather than service_account—and a file whose private_key isn't a PEM-encoded key. Paste the file exactly as downloaded, keeping the \n escapes intact.

The dialog rejects your project ID. Project IDs are 6 to 30 characters: lowercase letters, digits, and hyphens, starting with a letter and not ending in one. An all-digits value is the project number, which the Google Cloud console shows next to the ID; the gateway needs the ID.

A request fails with a 403 about the Vertex AI API. The API isn't enabled on the project. Enable Vertex AI API under APIs & Services → Library and confirm the project has billing enabled. This error names the API rather than permissions, so it's easy to mistake for a credential problem.

A request fails with a permission error. The service account is missing the Vertex AI User (roles/aiplatform.user) role, or holds it on a different project than the one configured on the provider. Grant the role on the same project you entered as the Google Cloud Project ID.

A request fails with 400 saying the model isn't available in the region, or that a Claude model "is not servable in region". The model isn't served from your provider's location, or isn't enabled for that location in your project. Most Model Garden models are global-only, and Claude models are served from global and the regions on each model's card—see Model availability. Set the provider's Location to global, or use a provider that points at a location serving that model.

An embeddings request fails with 400 about the input. The gateway accepts at most 250 inputs per request and text input only. See Embeddings.

A /v1/responses request fails with 400. Vertex serves the OpenAI Responses API for none of its models. Use /v1/chat/completions, or another provider—see Supported endpoints.

A /v1/messages request for a Gemini or Model Garden model fails with 400. Only Claude models serve the Messages API on Vertex. Use /v1/chat/completions for the others.

A Claude model fails with 400 on /v1/chat/completions. Claude models are Messages-only on Vertex. Send the request to /v1/messages with an Anthropic client—see Call Claude models on the Messages API.

A Claude request fails with 404 saying the model "was not found or your project does not have access to it". Either the model isn't enabled in Model Garden for your project (see Set up Google Cloud) or the model ID is wrong. Enabling a model applies to every location; a location that doesn't serve the model is the separate 400 described above.

A Claude request fails with 403 about data sharing. Some Claude models require the project to turn on data sharing for the anthropic publisher. The error names the setPublisherModelConfig setting to change.

A request fails when you use @ai-sdk/google-vertex. The provider posts to Vertex's own publisher paths—{model}:generateContent for Gemini, and {model}:rawPredict on its /anthropic entry—which the gateway doesn't serve, so both return a 404. Use @ai-sdk/openai-compatible for Gemini and Model Garden models and @ai-sdk/anthropic for Claude—see the AI SDK Vertex section. For its /anthropic (Claude) entry specifically, the SDK path shim can repoint @ai-sdk/google-vertex/anthropic onto /v1/messages; the Gemini entry can't be repointed that way.

A request fails when you use @anthropic-ai/vertex-sdk. The AnthropicVertex client rewrites the request onto Vertex's publishers/anthropic/models/{model}:rawPredict path, which the gateway doesn't serve. Use the plain Anthropic client from @anthropic-ai/sdk with authToken—see Call Claude models on the Messages API. To keep using AnthropicVertex instead, install the SDK path shim, which rewrites that publisher path onto /v1/messages.

A request fails when you use @google/genai. With vertexai: true the client posts to Gemini's native {model}:generateContent path, and it also misreads a two-slash model reference as publisher/model. Use an OpenAI client on /v1/chat/completions—see Call the models.

Next steps

• AI Providers—the capability matrix across every supported provider.
• Universal API—the endpoints every app serves and how model references work.
• Managing Providers—edit models, keys, the location, and the project, and understand when changes deploy.
• AI Gateway Apps—create the apps that call your Vertex-backed models.
• Model Filtering policy—control which models each app can call.