# Using Bedrock Mantle **Bedrock Mantle** is [Amazon Bedrock's compatible-APIs endpoint](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html). One regional endpoint (`https://bedrock-mantle..api.aws`) and one long-term Bedrock API key serve models from many vendors—the Claude family, OpenAI's GPT-5.x and GPT-OSS families, Gemma, Qwen, DeepSeek, Mistral, Grok, Kimi, GLM, and more. Adding Bedrock Mantle as a provider gives your [apps](./apps.mdx) all of them through the [Universal API](./universal-api.mdx) with a single credential, on your AWS account and AWS billing. Mantle is the part of Amazon Bedrock's catalog that AWS exposes over these compatible APIs. A model that Bedrock offers only through its native `InvokeModel` or `Converse` APIs isn't served by Mantle, so the gateway can't route to it. Apps reference the models as `providerName/model`, where `providerName` is the name you give the provider configuration. A provider named `bedrockmantle` serves `bedrockmantle/anthropic.claude-sonnet-5` and `bedrockmantle/openai.gpt-oss-120b`. ## How the gateway routes Mantle models Mantle serves two API formats on the same host with the same API key: - An **OpenAI-compatible API**, serving every non-Claude model with OpenAI-style chat completions and—for the models that support it—the OpenAI Responses API. - The **native Anthropic Messages API**, serving the Claude models. Your app AI Gateway OpenAI-compatible API Anthropic Messages API The gateway's model catalog records which API serves each model, and the gateway routes every request accordingly—your clients always call your app's URL and never see the Mantle endpoint. This also covers a Mantle quirk: AWS serves some models (the Gemma 4 and GPT-5.x families among them) on a different base path (`/openai/v1` instead of `/v1`), documented per [AWS model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards.html)—the per-model reference pages in the AWS Bedrock docs. The gateway sends each model to its documented path, so the model reference and your app's URL stay the same either way. ## Supported endpoints by model family | Endpoint | OpenAI-compatible models | Claude models | | ---------------------- | ---------------------------- | ------------- | | `/v1/chat/completions` | ✅ Forwarded | ✅ Translated | | `/v1/responses` | ✅ Models that serve it | ❌ | | `/v1/messages` | ❌ | ✅ Native | | `/v1/embeddings` | ❌ Mantle has no such models | ❌ | Streaming (`stream: true`) works on all three serving endpoints. On the OpenAI-compatible side, endpoint support is per model, decided by AWS: - The GPT-5.x models (`openai.gpt-5.6-sol`, `openai.gpt-5.5`, and family) serve the Responses API only. Sending one to `/v1/chat/completions` returns Mantle's own error that the model doesn't support that API. - The GPT-OSS models (`openai.gpt-oss-120b`, `openai.gpt-oss-20b`) serve both chat completions and the Responses API. - Most of the other models serve chat completions only, and answer `/v1/responses` with an error. The **APIs supported** row of a model's AWS model card is the reference. When a model doesn't serve the requested API, the gateway forwards Mantle's error to your client. Claude models work on `/v1/chat/completions` through the gateway's translation to the Messages API, which supports the core chat parameters: `messages`, `max_tokens` (defaults to 1024 when omitted), `temperature`, `top_p`, `stop`, and `stream`. Other OpenAI parameters—`tools`, `response_format`, penalties, and the rest—are dropped before forwarding, and `n > 1` is rejected. For tool use or other Anthropic-specific features with Claude models, call [`/v1/messages`](#call-claude-models-on-the-messages-api) instead. ## Before you begin You need: - An AWS account with access to Amazon Bedrock in the region you plan to use. Model availability varies by region, and some models—the Claude Mythos models, for example—require an access grant from AWS. Check each model's AWS model card. - A long-term Bedrock API key. In the AWS console, open **Bedrock → API keys** and generate a long-term key—it starts with `ABSK`. - An AI Gateway project in the Zuplo Portal. - An AI Gateway [app](./apps.mdx) to call the models from. The app page shows the app's API URL, and its API key lives on the app's **API Key** tab. :::caution{title="Long-term API keys only"} Bedrock issues both long-term and short-term API keys, and the gateway accepts only long-term ones, which start with `ABSK`. A short-term key (prefixed `bedrock-api-key-`) expires within 12 hours and is locked to the session that minted it, so the provider dialog rejects it rather than letting the connection break the same day. ::: ## Add the provider Adding or editing providers requires the **Edit** permission, granted to Zuplo account and project **Admins**—see [Managing Providers](./managing-providers.mdx). 1. Open [**Settings → AI Providers**](https://portal.zuplo.com/+/account/project/ai/settings/data-models) in your AI Gateway project in the Zuplo Portal. 1. Click the **Add Provider** button. 1. In the **AI Provider** list, select **Bedrock Mantle** from the Default Providers group. 1. Review the **Provider Name**, which fills in as `bedrockmantle` (a second configuration becomes `bedrockmantle-2`). You can replace it with your own name, but only now—the name is permanent after creation, and it's the prefix in every model reference: a provider named `bedrockmantle` serves `bedrockmantle/anthropic.claude-sonnet-5`. 1. In **AWS Region**, enter the lowercase code of the AWS region you use Bedrock in, such as `us-east-1`. The gateway sends this provider's requests to `https://bedrock-mantle..api.aws`—there's no endpoint URL to enter. 1. In **API Key**, paste your long-term Bedrock API key. The dialog rejects short-term keys and any value that doesn't start with `ABSK`. 1. Select the models to enable, or click **Select All**. The picker shows AWS's per-token prices, which the gateway uses to track cost per app. You can change the selection later. 1. Click **Create**. :::note Saving provider settings triggers an automatic production deployment of your gateway, because provider credentials are part of the deployed gateway. The change is live once the deployment completes. ::: When you edit the provider later—see [Managing Providers](./managing-providers.mdx)—the **AWS Region** field shows the saved region and stays editable, and you can replace the API key. The **Provider Name** doesn't change. ## Verify the provider Once the deployment completes, send a chat completions request to your app's [Universal API](./universal-api.mdx) URL—shown at the top of the [app page](./apps.mdx)—with the app's API key as the bearer token and one of the models you enabled: ```bash curl https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1/chat/completions \ -H "Authorization: Bearer $ZUPLO_APP_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "bedrockmantle/openai.gpt-oss-120b", "messages": [{ "role": "user", "content": "Say hi" }] }' ``` The URL is a sample—replace it with your app's API URL plus `/v1/chat/completions`, set `ZUPLO_APP_API_KEY` to the app's API key, and substitute any model you enabled for `bedrockmantle/openai.gpt-oss-120b`. A `200` response confirms the provider works; the body carries `"provider": "bedrock-mantle"`—the provider type, not the name you chose. If the request fails immediately after you save the provider, the deployment may not have finished—retry before debugging further. ## Call OpenAI-compatible models Use any OpenAI client with your app's URL plus `/v1` as the base URL and the app's API key: ```ts import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.ZUPLO_APP_API_KEY, baseURL: "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e/v1", }); const response = await client.chat.completions.create({ model: "bedrockmantle/openai.gpt-oss-120b", messages: [{ role: "user", content: "Summarize this ticket." }], }); ``` The same call works with a Claude model string—the gateway translates it to the Messages API—within the [translation's parameter subset](#supported-endpoints-by-model-family). ### Use the Responses API Models that serve the Responses API (the GPT-OSS and GPT-5.x families) work through `/v1/responses`, including streaming: ```ts const response = await client.responses.create({ model: "bedrockmantle/openai.gpt-oss-20b", input: "Write a haiku about container ships.", max_output_tokens: 512, }); ``` The GPT-OSS models reason before they answer, so give Responses calls a generous `max_output_tokens`—a small cap can complete the response with an empty `output`. Setting `store: true` persists the response into Bedrock's session storage, which is what makes the management operations—`GET /v1/responses/{id}`, `DELETE /v1/responses/{id}`, and `GET /v1/responses/{id}/input_items`—useful. Two Mantle-specific notes on them: - The management operations carry no request body, so the gateway can't read a model from the request. They work on apps whose [policy chain](./policy-chains.mdx) includes a [Model Filtering](../policies/ai-gateway-model-filtering-v2-inbound.mdx) policy with an allow list (the gateway routes them with the first listed model); without one they fail with a `400` error. - Whether `DELETE` succeeds is a permission of your Bedrock API key (the `bedrock-mantle:DeleteInference` action in AWS IAM—see [AWS's Bedrock Mantle documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html)). When the key lacks it, the gateway forwards Mantle's `401` response. ## Call Claude models on the Messages API Claude models serve the native [Anthropic Messages API](./universal-api.mdx#supported-endpoints) at `/v1/messages`. With the Anthropic SDK, set `baseURL` to the app's URL _without_ `/v1`—the SDK appends `/v1/messages` itself—and pass the app's API key as `authToken`, not `apiKey`: ```ts import Anthropic from "@anthropic-ai/sdk"; const client = new Anthropic({ baseURL: "https://my-gateway-main-2e18f50.zuplo.app/config_fe0a04972d2848e0a94ae4b8bcd1497e", authToken: process.env.ZUPLO_APP_API_KEY, // apiKey would send x-api-key, which the gateway ignores }); const message = await client.messages.create({ model: "bedrockmantle/anthropic.claude-sonnet-5", max_tokens: 1024, messages: [{ role: "user", content: "Say hi" }], }); ``` The gateway forwards the request body to Mantle verbatim, so everything the Messages API supports—tool use, system prompts, streaming—works as it does against Anthropic directly. Client headers such as `anthropic-beta` pass through, and `anthropic-version` defaults to `2023-06-01` when your client doesn't send one. ## Streaming details Most clients never notice these, but if you parse the SSE stream yourself: - Claude models on `/v1/chat/completions` stream gateway-translated `chat.completion.chunk` events ending with `data: [DONE]`. - OpenAI-compatible models stream exactly what Mantle sends. The GPT-OSS stream ends on a usage chunk with no `[DONE]` sentinel, so don't wait for one. - `/v1/messages` streams Anthropic's standard SSE events (`message_start` through `message_stop`). The gateway meters usage and cost on all of them, streamed or not. ## Troubleshooting **The dialog rejects your API key.** Only long-term Bedrock API keys work. A key starting with `bedrock-api-key-` is short-term—it expires within 12 hours. Generate a long-term key (starting with `ABSK`) in the AWS console under **Bedrock → API keys**, and paste it without surrounding whitespace. **A model returns `400` naming `/v1/messages` or `/v1/responses`.** The model's family doesn't serve that endpoint: the gateway rejects an OpenAI-compatible model on `/v1/messages`, or a Claude model on `/v1/responses`, before any upstream call. The error names the provider type (`bedrock-mantle`), which serves both APIs, so it can be confusing—support is per model family, per the [endpoint table](#supported-endpoints-by-model-family). **Mantle answers that the model doesn't support the API.** Endpoint support is per model on the OpenAI-compatible side—the GPT-5.x models serve only `/v1/responses`, and most other models serve only `/v1/chat/completions`. Check the **APIs supported** row on the model's AWS model card. **A model in the picker fails at request time.** The model picker lists Mantle's full catalog, but availability is per region and some models are access-gated by AWS. If a request fails upstream with a model-not-found or access error, check the model's AWS model card for regional availability, and your AWS account's model access for gated models such as the Claude Mythos family. **Embeddings requests fail.** Mantle serves no embeddings models, so the model picker lists none and `/v1/embeddings` has no Bedrock Mantle models to route to. Use another provider for embeddings. ## Next steps - [AI Providers](./providers.mdx)—the capability matrix across every supported provider. - [Universal API](./universal-api.mdx)—the endpoints every app serves and how model references work. - [Managing Providers](./managing-providers.mdx)—edit models, keys, and the region, and understand when changes deploy. - [AI Gateway Apps](./apps.mdx)—create the apps that call your Mantle-backed models. - [Model Filtering policy](../policies/ai-gateway-model-filtering-v2-inbound.mdx)—control which models each app can call.