OpenTelemetry
Configure OpenTelemetry destinations, content capture, and third-party exports.
eve dev records local traces by default. Use otel() for process-wide
settings and otelIntegration() for third-party destinations.
Configure OpenTelemetry
otel() and otelIntegration() are optional:
- Use
otel()only when you need process-wide settings such as the resource, sampler, propagators, or trace capture policy. Declare it at most once. - Use
otelIntegration()only when adding a third-party OpenTelemetry destination. Declare one file per exporter or processor chain.
Add agent/instrumentation/otel.ts when you need process-wide settings or
want to control the content eve writes to OpenTelemetry spans:
The OpenTelemetry tracePolicy is a capture ceiling shared by every OTel
destination. A destination cannot restore content excluded by this policy. It
does not affect lifecycle instrumentation created with defineInstrumentation().
See Audience for how channels classify the
conversation passed to this policy.
Instrument Node.js dependencies
Pass OpenTelemetry instrumentation instances to otel({ instrumentations }).
For example, install the PostgreSQL instrumentation alongside pg:
Keep the instrumented package external so Node.js module hooks can intercept
it. Add pg to your agent's build configuration:
Add an OpenTelemetry destination
Use this section when sending traces to a third-party destination. Install
@vercel/otel before using its exporters:
Pass spanProcessors to otelIntegration() when a custom destination needs
filtering or transformation. The example's exportPolicy preserves span
metadata while removing inputs and outputs from this destination.
Add runtime context
Use runtimeContext to add JSON values to AI SDK spans for each model attempt.
For typed channel metadata, import the channel definition and narrow with
isChannel:
The resolver receives the channel, session, final model input, step, and turn.
Keys beginning with eve. are reserved. Content excluded by
otel({ tracePolicy }) is not available to the resolver.
Authored values appear under ai.settings.context.*. Framework channel, session,
step, turn, and version identifiers use structural span attributes instead of
runtime-context copies. ai.settings.context.eve.environment remains available.
Trace topology
OpenTelemetry destinations receive the following trace topology for an ordinary agent turn:
Each root agent turn starts a new trace. If eve receives a local caller's trace context, the first local subagent turn uses that trace. Its span is a child of the dispatch span and has a different span ID. Without caller trace context, the first subagent turn starts a new trace. Later turns in the child session start new traces. They do not use the original dispatch span as their parent.
Use gen_ai.tool.call.id to correlate tool and approval spans. Tool failures use
error.type. These attributes replace the duplicate agent.action.call_id and
agent.action.error.code fields without changing trace schema version 4.
Model and activation spans record token counts under gen_ai.usage.*, without
duplicate agent.usage.* token fields. Step and tool spans retain agent.usage.*.
Cost values remain unchanged.
A remote child starts a separate trace with a root invoke_agent span. Its first
turn links to the caller with eve.link.type=agent.dispatch. The receiving
deployment applies its sampler. Queries for local dispatches must use
parentSpanId or agent.parent_run.id and agent.parent_call.id instead of a
dispatch link.
Use gen_ai.conversation.id to find the root session and its child sessions
across traces. eve adds this ID to spans created during model, tool, and memory
execution. This includes spans from third-party instrumentation. agent.run.id
identifies one agent session, not the full conversation.
The shared conversation ID does not give access to message history, agent state,
or credentials. It does not give authorization or require the same worker for
each turn. Remote requests carry the conversation ID in W3C Baggage.
trustedForwarders controls whether the receiver accepts parent session data
and the trace-content policy. See
Remote agents.
Local traces
eve dev records local traces by default. Omitting
agent/instrumentation/local.ts preserves that default. Export
localTraces(...) from that file to reconfigure it, or disable it explicitly:
Enable tracing on Vercel
For Agent Runs sampling defaults and existing-deployment setup, see Enable tracing on Vercel.
Filter a destination
otelIntegration() and localTraces() accept one
exportPolicy object or an array applied in order. Each policy filters spans
and attributes before that destination's processors receive them:
Return { emit: true } from span to retain a span unchanged, or
{ emit: false } to omit it from this destination. To retain the span while
redacting content, return { redact: true } with inputs: true,
outputs: true, or both. A redaction decision implies emission and requires
at least one direction. For example, return
{ redact: true, inputs: true, outputs: true } to redact both directions.
A throwing span callback drops the span from this destination. Return
{ emit: true }, { emit: false }, or
{ replace: true, value } from attribute to retain, remove, or change
one attribute.
What to read next
- Instrumentation: handle eve lifecycle events.
- Agent Runs: configure Vercel's built-in trace destination.
- Migrate instrumentation: replace
agent/instrumentation.ts. - Local development: inspect local traces in the TUI.