Using Vertex AI

Vertex AI is Google Cloud's managed model platform. Adding it as a provider serves both the Gemini family and Vertex's Model Garden partner models—DeepSeek, Qwen, GLM, Kimi, MiniMax, Gemma, and GPT-OSS—to your apps through the Universal API, running on your Google Cloud project and your Google Cloud billing.

Chat completions go to Vertex's OpenAI-compatible endpoint, so the gateway forwards OpenAI-shaped requests without translating them. Embeddings work too, though Vertex has no OpenAI-compatible embeddings endpoint—the gateway translates those onto Vertex's own embedding APIs, which is invisible from your app. What sets this provider apart is how you connect it: the endpoint is per-location, the Google Cloud project is a separate setting, and the credential is a service account key file—not an API key.

Model references include the publisher prefix

Apps reference a provider's models as providerName/model. Every Vertex model ID is publisher-qualified, so a Vertex model reference has two slashes in it:

Code
vertexai/google/gemini-3.7-flash
└──┬───┘ └──────────┬──────────┘
provider          model

Everything before the first slash is the provider name you chose, and everything after it is the model ID exactly as Vertex expects it. The publisher isn't always google—Model Garden models carry their own vendor's prefix. A provider named vertexai serves all of these:

• vertexai/google/gemini-3.7-flash
• vertexai/qwen/qwen3-coder-480b-a35b-instruct-maas
• vertexai/deepseek-ai/deepseek-v3.2-maas
• vertexai/zai-org/glm-5-maas

Copy model IDs from the model picker rather than typing them—the -maas suffix and the publisher prefix are both part of the ID.

How the gateway calls Vertex AI

When no valid cached token exists, the gateway uses the service account key to request a short-lived Google OAuth 2.0 access token. It caches and refreshes that token, then calls Vertex's OpenAI-compatible endpoint with the token as the bearer credential.

The request URL the gateway builds combines both settings you enter:

Code
https://us-central1-aiplatform.googleapis.com
  /v1/projects/my-project-123/locations/us-central1/endpoints/openapi/chat/completions

The location lives in the hostname, which is why the provider dialog asks for a location instead of an endpoint URL. The project doesn't appear in the hostname at all—it appears only in the path—so it can't be derived from the endpoint and is a field of its own. That's why the project is required rather than optional: without it the gateway would have to guess, and a guess would send your traffic and your spend to someone else's Google Cloud project.

Supported endpoints

Streaming (stream: true) works on chat completions.

The gateway rejects /v1/responses and /v1/messages requests for Vertex models with a 400 error before any upstream call, because Vertex serves neither the OpenAI Responses API nor the Anthropic Messages API.

Model availability depends on the location

Google registers each Vertex model in specific locations, and it changes that list over time. Most Model Garden models are served only from the global location; the Gemini models are more widely available.

The model picker doesn't filter by your provider's location, so every model stays selectable from every provider. If you call a model that isn't served where your provider points, Vertex answers with a 400 that names both:

Code
Publisher model 'publishers/qwen/models/qwen3-coder-480b-a35b-instruct-maas'
is not available in region 'us-central1'

Set the provider's Location to global if you plan to use Model Garden models, or add a second Vertex provider for a specific region and route to whichever one serves the model you want.

Before you begin

You need:

• A Google Cloud project with the Vertex AI API (aiplatform.googleapis.com) enabled and billing enabled. Gemini models require both.
• A service account in that project holding the Vertex AI User (roles/aiplatform.user) role, and a JSON key for it. The steps are in Set up Google Cloud.
• The project's project ID—the readable identifier such as my-project-123, not the numeric project number.
• An AI Gateway project in the Zuplo Portal.
• An AI Gateway app to call the models from. The app page shows the app's API URL, and its API key lives on the app's API Key tab.

Set up Google Cloud

Do this before adding the provider—the key file you download in the last step is what the provider dialog asks for.

Add the provider

Adding or editing providers requires the Edit permission, granted to Zuplo account and project Admins—see Managing Providers.

When you edit the provider later—see Managing Providers—the Location and Google Cloud Project ID fields show the saved values and stay editable, and you can replace the key file. Leave the key field blank to keep the stored key. The Provider Name doesn't change.

Verify the provider

Once the deployment completes, send a chat completions request to your app's Universal API URL—shown at the top of the app page—with the app's API key as the bearer token and one of the models you enabled:

Code
curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions \
  -H "Authorization: Bearer $ZUPLO_APP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertexai/google/gemini-3.7-flash",
    "messages": [{ "role": "user", "content": "Say hi" }]
  }'

The URL is a sample—replace it with your app's API URL plus /v1/chat/completions, set ZUPLO_APP_API_KEY to the app's API key, and substitute any model you enabled for vertexai/google/gemini-3.7-flash.

A 200 response confirms the provider works; the body carries "provider": "vertex-ai"—the provider type, not the name you chose. If the request fails immediately after you save the provider, the deployment may not have finished—retry before debugging further.

Call the models

Use any OpenAI client with your app's URL plus /v1 as the base URL and the app's API key:

Code
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ZUPLO_APP_API_KEY,
  baseURL:
    "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1",
});

const response = await client.chat.completions.create({
  model: "vertexai/google/gemini-3.7-flash",
  messages: [{ role: "user", content: "Summarize this ticket." }],
});

Streaming works the same way—set stream: true and read the chat.completion.chunk events.

Embeddings

Embeddings use the same client and the same /v1/embeddings endpoint as any other provider:

Code
const embeddings = await client.embeddings.create({
  model: "vertexai/google/gemini-embedding-2",
  input: ["first document", "second document"],
});

dimensions and encoding_format both work. Two Vertex-specific limits apply, and the gateway enforces both before calling Vertex, so you get a 400 rather than a partial charge:

• At most 250 inputs per request. Split larger batches yourself.
• Text input only. Pre-tokenized input—arrays of token IDs, which the OpenAI API also accepts—is rejected, because Vertex has no way to represent it. Send the original text.

Troubleshooting

The dialog rejects your credential. Vertex uses a service account, not an API key. The field takes the whole JSON key file downloaded from IAM & Admin → Service Accounts → Keys. The dialog also rejects a workload identity federation configuration—its type is external_account rather than service_account—and a file whose private_key isn't a PEM-encoded key. Paste the file exactly as downloaded, keeping the \n escapes intact.

The dialog rejects your project ID. Project IDs are 6 to 30 characters: lowercase letters, digits, and hyphens, starting with a letter and not ending in one. An all-digits value is the project number, which the Google Cloud console shows next to the ID; the gateway needs the ID.

A request fails with a 403 about the Vertex AI API. The API isn't enabled on the project. Enable Vertex AI API under APIs & Services → Library and confirm the project has billing enabled. This error names the API rather than permissions, so it's easy to mistake for a credential problem.

A request fails with a permission error. The service account is missing the Vertex AI User (roles/aiplatform.user) role, or holds it on a different project than the one configured on the provider. Grant the role on the same project you entered as the Google Cloud Project ID.

A request fails with 400 saying the model isn't available in the region. The model isn't served from your provider's location. Most Model Garden models are global-only—see Model availability. Set the provider's Location to global, or use a provider that points at a location serving that model.

An embeddings request fails with 400 about the input. The gateway accepts at most 250 inputs per request and text input only. See Embeddings.

A /v1/responses or /v1/messages request fails with 400. Vertex's OpenAI-compatible endpoint serves neither API. Use /v1/chat/completions, or another provider for those endpoints—see Supported endpoints.

Next steps

• AI Providers—the capability matrix across every supported provider.
• Universal API—the endpoints every app serves and how model references work.
• Managing Providers—edit models, keys, the location, and the project, and understand when changes deploy.
• AI Gateway Apps—create the apps that call your Vertex-backed models.
• Model Filtering policy—control which models each app can call.