OTLP Provider Guidance for Gateway Propagation

This guide documents how to point the LLM gateway at OTLP-compatible backends while preserving the gateway’s W3C propagation contract. For request lifecycle span and log names, see Gateway OTEL Lifecycle.

Propagation Contract (Backend Agnostic)

Gateway trace-context behavior is backend-agnostic and should remain consistent across OTLP providers:

• inbound proxy requests may include W3C traceparent, tracestate, baggage
• proxy server spans continue valid inbound lineage when context is valid
• malformed inbound trace context does not fail requests
• outbound proxy egress always injects gateway-owned W3C context
• stale outbound W3C headers are overwritten
• baggage propagation is fail-open:
  • valid baggage propagates
  • malformed baggage is dropped while request/trace propagation continues

Use just --justfile llm-gateway/justfile obs-otel-regression-check to run the local regression workflow that validates this contract.

OTLP Runtime Knobs

Common OTLP settings used by the gateway:

• OTEL_TRACES_EXPORTER, OTEL_LOGS_EXPORTER
• OTEL_EXPORTER_OTLP_ENDPOINT (or per-signal endpoints)
• OTEL_EXPORTER_OTLP_PROTOCOL (http/protobuf or grpc)
• OTEL_EXPORTER_OTLP_HEADERS (for auth tokens and tenant headers)
• OTEL_RESOURCE_ATTRIBUTES (for example deployment.environment=staging)

Splunk OTLP Example

Use this as a starting point and replace placeholder values:

export OTEL_TRACES_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<realm>.signalfx.com"
export OTEL_EXPORTER_OTLP_HEADERS="x-sf-token=<SPLUNK_ACCESS_TOKEN>"
export OTEL_RESOURCE_ATTRIBUTES="service.name=guardian-llm-gateway,deployment.environment=staging"

Notes:

• Splunk realm and token values are workspace-specific.
• Use per-signal endpoint/header variables if your environment requires split endpoints.
• Keep service.name stable across environments to preserve consistent service identity in traces/logs.