Copy page

AI Gateway

Galileo Tracing Policy

The Galileo Tracing policy sends AI Gateway requests, responses, token usage, and timing data to Galileo for observability and quality evaluation. It supports streaming and non-streaming traffic across all AI Gateway request shapes.

Place it near the end of the application policy chain so traces reflect the request body after earlier policy changes. Trace delivery runs in the background and does not modify the client response. Streaming output tracing is skipped if an earlier response hook has already consumed the original stream.

Configuration

The configuration shows how to configure the policy in the 'policies.json' document.

Code
{
  "name": "my-galileo-tracing-v2-inbound-policy",
  "policyType": "galileo-tracing-v2-inbound",
  "handler": {
    "export": "GalileoTracingV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "apiKey": "$env(GALILEO_API_KEY)",
      "projectId": "your-project-id",
      "logStreamId": "your-log-stream-id"
    }
  }
}

Policy Configuration

• name <string> - The name of your policy instance. This is used as a reference in your routes.
• policyType <string> - The identifier of the policy. This is used by the Zuplo UI. Value should be galileo-tracing-v2-inbound.
• handler.export <string> - The name of the exported type. Value should be GalileoTracingV2InboundPolicy.
• handler.module <string> - The module containing the policy. Value should be $import(@zuplo/runtime).
• handler.options <object> - The options for this policy. See Policy Options below.

Policy Options

The options for this policy are specified below. All properties are optional unless specifically marked as required.

• apiKey (required) <string> - The Galileo API key for authentication.
• projectId (required) <string> - The Galileo project ID (UUID) for organizing traces.
• logStreamId (required) <string> - The Galileo log stream ID (UUID) for organizing traces.
• baseUrl <string> - The base URL for the Galileo API (optional, defaults to https://api.galileo.ai).
• endpoints <string[]> - The endpoint shapes this policy applies to. Omit to apply to all (openai-chat, openai-responses, anthropic-messages).
• onUnknownShape <string> - What to do on a shape the policy cannot read. As an observer this defaults to 'skip' (fail open — pass through untraced). Allowed values are deny, skip. Defaults to "skip".

Using the Policy

Galileo Tracing

The Galileo Tracing policy integrates Galileo with the AI Gateway. It automatically records LLM inputs, outputs, model information, token usage, request metadata, and timing so you can debug applications and monitor quality in production.

The policy supports Chat Completions, Responses, and Anthropic Messages. For a streaming response, it observes a clone of the original stream in the background when that stream is still readable; the client does not wait for trace delivery. If an earlier response hook has already consumed the original stream, output tracing is skipped and the client response continues unchanged.

Configure the policy

Create a Galileo project and log stream, then generate an API key. The project ID and log stream ID appear in the Galileo log-stream URL. Declare the policy in config/policies.json and add it to each application chain that should emit traces:

Code
{
  "name": "galileo-tracing-v2-inbound",
  "policyType": "galileo-tracing-v2",
  "handler": {
    "export": "GalileoTracingV2InboundPolicy",
    "module": "$import(@zuplo/runtime)",
    "options": {
      "apiKey": "$env(GALILEO_API_KEY)",
      "projectId": "your-project-id",
      "logStreamId": "your-log-stream-id"
    }
  }
}

Place the policy near the end of the chain so the trace records request-body changes made by earlier policies. The recorded model label comes from the app configuration when available, then falls back to the request body. A model selected through request-scoped routing is not included in this trace label.

Options

Option	Default	What it does
apiKey	Required	Authenticates to Galileo.
projectId	Required	Selects the Galileo project.
logStreamId	Required	Selects the Galileo log stream.
baseUrl	https://api.galileo.ai	Overrides the Galileo API URL.
endpoints	All shapes	Limits tracing to selected AI Gateway endpoint shapes.
onUnknownShape	skip	Skips or denies a request whose shape cannot be read.

As an observer, the policy fails open by default. A Galileo API failure is logged but does not fail the AI request. Setting onUnknownShape to deny only changes the behavior for an uninspectable request shape.

Trace structure

Each request produces a hierarchy that Galileo can analyze:

1. A top-level trace represents the complete AI Gateway request.
2. A workflow span represents gateway processing.
3. An LLM span records the provider interaction.

Records include prompt and completion text, model parameters, finish reasons, input and output token counts, duration, Zuplo request ID, and route. This data supports trace inspection, model comparison, quality evaluation, cost analysis, and production monitoring.

Additional resources

• Galileo documentation

Read more about how policies work