Gateway OTEL Lifecycle

The LLM gateway emits a lifecycle trace and matching OTEL log events for LLM proxy requests. The contract is intentionally request-centric: the root span is the full gateway request, child spans represent lifecycle stages, and lifecycle log events carry event.sequence so log backends can reconstruct in-request order even when async plugin work is involved.

Instrumentation name for lifecycle traces and logs:

guardian.gateway.lifecycle

Span Shape

gateway.request
  gateway.request.prepare
  gateway.plugin.invocation     # request.start, when matching plugins run
  gateway.verification
  gateway.proxy
    gateway.plugin.invocation   # proxy.pre, when matching plugins run
    gateway.upstream
      gateway.plugin.invocation # proxy.post, when matching plugins run

The root gateway.request span includes viper.state_cid when the inbound request provides X-Viper-State-CID.

request.end plugins are asynchronous. Their queued work carries W3C trace propagation and a reserved event.sequence value into the worker so worker plugin logs continue after the synchronous queue/request-end events. Worker plugin invocation spans are emitted as gateway.plugin.invocation spans with hook.name=request.end.

Lifecycle Logs

Every lifecycle log has correlation_id. Logs also include event.name as a durable attribute because some local Collector/file-exporter versions omit the OTEL log record’s top-level event name field. Logs emitted on the synchronous request path also carry a monotonic event.sequence for that request. Async request.end plugin worker logs continue from the reserved sequence captured when the queue item was written.

gateway.request.start
  attrs: correlation_id, viper.state_cid, http.method, gateway.uri, llm.provider, llm.operation

gateway.request.prepare.start
  attrs: correlation_id
gateway.request.prepare.end
  attrs: correlation_id, llm.request_model, llm.stream_requested, gateway.route_supported, upstream.uri

gateway.verification.start
  attrs: correlation_id, viper.state_cid
gateway.verification.end
  attrs: correlation_id, agent.id, signer.key_id, verification.outcome, verification.decision_code

gateway.proxy.start
  attrs: correlation_id, agent.id, llm.provider, llm.operation, upstream.uri

gateway.plugin.invocation.start
  attrs: correlation_id, hook.name, plugin.id, plugin.mode, plugin.runtime, snapshot.version, plugin.order_index
gateway.plugin.invocation.end
  attrs: correlation_id, hook.name, plugin.id, plugin.outcome, plugin.decision_code, plugin.status_code, plugin.error_code, plugin.latency_ms
gateway.plugin.invocation.queued
  attrs: correlation_id, hook.name=request.end, snapshot.version

gateway.upstream.start
  attrs: correlation_id, llm.provider, llm.operation, upstream.uri
gateway.upstream.end
  attrs: correlation_id, http.status_code, llm.response_model, llm.stream_observed

gateway.proxy.end
  attrs: correlation_id, gateway.route_type, policy.decision_code, http.status_code

gateway.request.end
  attrs: correlation_id, agent.id, gateway.route_type, policy.decision_code, http.status_code, gateway.duration_ms

Error And Deny Semantics

Verification denials use verification.outcome=denied. Internal verification failures use verification.outcome=error.

Plugin denials are policy outcomes, not span errors. gateway.plugin.invocation spans are marked with OTEL error status only for plugin execution or context failures, such as timeouts, runtime errors, and invalid context writes.

Proxy and request completion severities follow the response outcome:

• INFO for successful proxy responses.
• WARN for local denials and other 4xx outcomes.
• ERROR for upstream failures and 5xx outcomes.

Payload Policy

Lifecycle OTEL logs do not capture request bodies, response bodies, or raw headers. Payload inspection remains in the audit subsystem and plugin previews, where retention and redaction are controlled separately.