AI Gateway

Comet Opik Tracing Policy

AI Gateway Policy

This policy is for use with the AI Gateway. See the AI Gateway documentation to learn how to configure and govern AI models with Zuplo.

The Comet Opik Tracing policy sends AI Gateway requests, responses, token usage, and timing data to Comet Opik for observability and evaluation. It supports streaming and non-streaming traffic across all AI Gateway request shapes.

Place it near the end of the application policy chain so traces reflect the request body after earlier policy changes. Creating the top-level trace is synchronous, so the provider request waits for the Opik API call to finish, including failed calls. Response-span delivery runs in the background and does not modify the client response. Streaming output tracing is skipped if an earlier response hook has already consumed the original stream.

Configuration

The configuration shows how to configure the policy in the 'policies.json' document.


Code
{
  "name": "my-comet-opik-tracing-v2-inbound-policy",
  "policyType": "comet-opik-tracing-v2-inbound",
  "handler": {
    "export": "CometOpikTracingV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "apiKey": "$env(COMET_OPIK_API_KEY)",
      "projectName": "your-project-name",
      "workspace": "your-workspace"
    }
  }
}

Policy Configuration

name <string> - The name of your policy instance. This is used as a reference in your routes.
policyType <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be comet-opik-tracing-v2-inbound.
handler.export <string> - The name of the exported type. Value should be CometOpikTracingV2InboundPolicy.
handler.module <string> - The module containing the policy. Value should be $import(@zuplo/runtime).
handler.options <object> - The options for this policy. See Policy Options below.

Policy Options

The options for this policy are specified below. All properties are optional unless specifically marked as required.

apiKey (required) <string> - The Comet Opik API key for authentication.
projectName (required) <string> - The Comet Opik project name for organizing traces.
workspace (required) <string> - The Comet Opik workspace name.
baseUrl <string> - The base URL for the Comet Opik API (optional, defaults to https://www.comet.com/opik/api).
endpoints <string[]> - The endpoint shapes this policy applies to. Omit to apply to all (openai-chat, openai-responses, anthropic-messages).
onUnknownShape <string> - What to do on a shape the policy cannot read. As an observer this defaults to 'skip' (fail open — pass through untraced). Allowed values are deny, skip. Defaults to "skip".

Using the Policy

Comet Opik Tracing

The Comet Opik Tracing policy integrates Comet Opik with the AI Gateway. It captures LLM inputs, outputs, model information, token usage, request metadata, and timing so you can debug applications, monitor production traffic, and evaluate model output in Opik.

The policy supports Chat Completions, Responses, and Anthropic Messages. It creates the top-level trace synchronously before forwarding the request, so Opik API latency delays the provider request even when trace creation fails. The response span is delivered in the background. For a streaming response, the policy observes a clone of the original stream when it is still readable; the client does not wait for response-span delivery. If an earlier response hook has already consumed the original stream, output tracing is skipped and the client response continues unchanged.

Configure the policy

Create or select an Opik project and workspace, then generate an API key. Declare the policy in config/policies.json and add it to each application chain that should emit traces:


Code
{
  "name": "comet-opik-tracing-v2-inbound",
  "policyType": "comet-opik-tracing-v2",
  "handler": {
    "export": "CometOpikTracingV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "apiKey": "$env(COMET_OPIK_API_KEY)",
      "projectName": "your-project-name",
      "workspace": "your-workspace"
    }
  }
}

Place the policy near the end of the chain so the trace records request-body changes made by earlier policies. The recorded model label comes from the app configuration when available, then falls back to the request body. A model selected through request-scoped routing is not included in this trace label.

Options

Option	Default	What it does
`apiKey`	Required	Authenticates to Comet Opik.
`projectName`	Required	Selects the Opik project that receives traces.
`workspace`	Required	Selects the Opik workspace.
`baseUrl`	`https://www.comet.com/opik/api`	Overrides the Opik API URL.
`endpoints`	All shapes	Limits tracing to selected AI Gateway endpoint shapes.
`onUnknownShape`	`skip`	Skips or denies a request whose shape cannot be read.

As an observer, the policy fails open by default. A trace API failure is logged but does not fail the AI request. Setting onUnknownShape to deny only changes the behavior for an uninspectable request shape.

Trace structure

The policy creates one top-level trace when the request begins and an LLM span when the response completes. Records include:

Prompt or input content and the model name.
Completion text for streaming and non-streaming responses.
Input, output, and total token counts when the provider reports them.
Zuplo request ID and route for correlation.
Temperature and maximum-token settings when present.

In Opik, you can annotate these traces, group them into datasets, run heuristic or LLM-as-a-judge evaluations, compare model variants, and monitor quality and cost trends.

Additional resources