Pick when to use a span attribute versus a span event
Attribute = property of the span (indexed, filterable). Event = time-ordered occurrence inside the span (model name attribute; streamed-chunk event).
Think of a single page in a diary about one day. The date, the weather, and the place at the top of the page are attributes: they describe the whole day, and you can flip through your diary by date or weather to find similar days. The bullet points down the page that say what happened at 9am and 11am and 3pm are events: they are time-ordered things that happened during the day. You would not put 'lunch at noon' in the page header, and you would not write the date next to every event line. The two slots have different jobs. Span attributes and span events split the same way.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
OpenTelemetry gives spans two ways to attach data: attributes and events. They are structurally similar (both are key value ish payloads on a span) but they exist for different purposes, and observability backends treat them differently. Confusing them is one of the most common instrumentation bugs in LLM applications, where multi-message chats and streaming outputs make the choice feel ambiguous when it actually is not.
This walkthrough explains what each slot is for, the cardinality and indexing argument that drives the split, the specific decisions in the 2026 OTel GenAI semantic conventions, the failure modes when the split is done wrong, and the practical patterns for handling messages, streaming, retries, and PII.
Mental model: attribute is an adjective describing the span. Event is a verb describing something that happened during the span. Backends index adjectives for fast filtering; they timestamp verbs for the span timeline view.
The definitions and the cardinality argument
Span attribute
A key-value property of the span as a whole. Each attribute has a key (gen_ai.request.model) and a value ('gpt-5.5'). The full set of attributes describes the span's static metadata: what it was, who triggered it, what it returned.
Attributes are indexed by the observability backend. That index is what makes filtering ('show me all chat spans where model=gpt-5.5') and grouping ('aggregate cost by user.tier') fast.
Span event
A time-ordered record nested inside the span timeline. Each event has its own timestamp, a name (gen_ai.user.message, retry.attempt), and an optional attribute payload. A span can have zero or many events; their order is preserved.
Events are searchable but not pre-aggregated the same way attributes are. The span timeline view renders them as inline annotations.
The cardinality argument
Attribute keys live in the backend's index. The index has cost (memory, storage, query latency). Backends like Honeycomb, Datadog, and Tempo cap or price by attribute-key cardinality.
- Low-cardinality attribute keys (tens of distinct values across the system): cheap, fine.
- High-cardinality attribute keys (millions of distinct values): expensive, often capped.
- Unbounded attribute keys (new key per span, like
message.1,message.2, ...): index disaster.
The rule that emerges: attributes should be span-wide and low to medium cardinality. Anything per-occurrence or unboundedly variable goes in events.
The indexability argument
Flipping the other way: events are not indexed for filtering the same way attributes are. Hiding model_name inside an event means you cannot filter dashboards by model. The convenience of putting everything in events ('it is more flexible') is exactly the cost: the data is no longer queryable in the fast path.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenTelemetry GenAI semantic conventions explicitly use events for messages and attributes for per-span metadata.
- Langfuse 3.x, Arize Phoenix 5.x, and Datadog LLM Observability all index on attributes and render events on the span timeline.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you record retry attempts on an LLM call?
Each retry is a time-ordered occurrence inside the parent span; use events with names like retry.attempt and payload {attempt_number, error_code, backoff_ms}. Aggregate retry count as a derived attribute gen_ai.retries.total for filtering.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Recording per-message data as attributes. Cardinality explodes and the index breaks; messages belong in events.
60 second bullets to scan on the way to the call.
Definition of span attribute vs span event
Cardinality and indexing as the reason for the split
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.