Describe how gen_ai.operation.name separates chat, completion, and embedding spans

Flashcard·Medium·4.0 · 0·~30s·Asked atDatadog Ola Vellum

Attempt it

Your answer

The OpenTelemetry GenAI semantic conventions standardize the names of common attributes across LLM observability vendors. `gen_ai.operation.name` is one of those names. It is a **low-cardinality enum** that labels each generation span with what kind of operation it represents. The canonical values include: - `chat`: a multi-turn chat completion (the dominant LLM call shape today). - `text_completion`: a single-shot completion (legacy GPT-3 style). - `embeddings`: a vector-embedding call. - `image_generation`, `audio_transcription`, and a few others depending on provider. Why it matters for dashboards: these operations have wildly different latency and cost baselines. A chat call to GPT-5.5 can take 2 to 10 seconds; an embedding call to text-embedding-3-large can return in 50 milliseconds. If your dashboard computes 'p99 latency across all GenAI spans', the chat distribution dominates but the embedding traffic skews the lower percentiles. The number you report is meaningless for either workload. Grouping by `gen_ai.operation.name` fixes this. You get one chart per operation type with its own baseline. The same attribute also drives sampling: most stacks sample embeddings down to 1 percent (high volume, low cost per trace) and keep chat at 100 percent (low volume, high cost per trace).

**`gen_ai.operation.name` is the OTel GenAI attribute that labels each span with the kind of LLM operation it represents.** Values are low-cardinality and enumerated: `chat`, `text_completion`, `embeddings`, `image_generation`, `audio_transcription`, and a few others per provider. The low cardinality is deliberate. Attributes used for grouping in time-series dashboards need to be finite-cardinality (10s of distinct values, not millions). User ids would explode storage; operation type is naturally small. **Why it matters for dashboards.** Each operation has its own latency, cost, and reliability profile: - Chat: 1 to 10 seconds, dollars per million tokens, the bulk of cost spend. - Embeddings: 30 to 200 milliseconds, cents per million tokens, often 10x to 100x the volume of chat. - Text completion: similar to chat but increasingly rare in 2026. - Image generation: 2 to 30 seconds, priced per image. If you compute aggregate p99 latency across all spans, the embedding spans skew the lower percentiles down; the chart no longer reflects the user-facing chat experience. Grouping by `gen_ai.operation.name` gives you one per-operation series with a meaningful baseline. **Why it matters for sampling.** Sampling rates differ by operation. Embeddings are cheap and high-volume; 1 percent sampling is typical. Chat is expensive and lower-volume; 100 percent sampling is typical. Tail-based samplers use the operation name as a primary key. **Why it matters for cost attribution.** Per-operation cost rolls up cleanly: 'embedding spend' vs 'chat spend' vs 'image-generation spend'. A 30 percent month over month cost increase decomposes by operation, so you know which workload is responsible. It is one of the most useful attributes in the OTel GenAI spec because it is small, stable, and load-bearing across all three downstream workflows (dashboards, sampling, cost).

TL;DR

Low-cardinality enum (chat, text_completion, embeddings) used to group dashboards and route sampling policies per operation type.

Memory aid

Easy to grasp

Picture a hospital that posts one big sign with the average wait time across every kind of visit, mixing dental cleanings with brain surgeries. The number is useless because the visits are not comparable. Now imagine the hospital paints a label on every patient's wristband saying what kind of visit they came for: cleaning, X-ray, surgery. Suddenly the front desk can post a separate wait time for each kind, and an unusual delay actually means something because you're comparing like with like. We do the same trick when a program talks to an AI. We stamp each request with a short label like chat, embedding, or completion. Then the dashboards group them by label, and the people on call only get woken up when the right kind of wait actually grows.

Key concepts

Observability & Tracing

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

OpenTelemetry GenAI semantic conventions are the OTel community's attempt to keep LLM observability vendors from inventing incompatible attribute shapes. The conventions define stable, low-cardinality names for the dimensions every LLM span carries: provider, model, token counts, finish reasons, and operation type. Of these, gen_ai.operation.name is one of the most useful because it carries dashboard partitioning, sampling routing, and cost attribution all on one tiny enumerated attribute.

This walkthrough explains what the enum values are, why low cardinality is a hard design requirement, what dashboards, samplers, and cost reports actually do with the attribute, and the discipline issues (model-id conflation, multi-modal turns) that come up in real production usage.

Mental model: gen_ai.operation.name is the outermost grouping key for every LLM-observability workflow. Get it right and dashboards, sampling, and cost reports all work. Get it wrong (or skip it) and each workflow degrades in its own way.

The enum and why it is small

Canonical values

The OTel GenAI spec defines a small enumerated set:

chat: multi-turn chat completion. The dominant shape in 2026.
text_completion: legacy single-shot completion. Increasingly rare.
embeddings: vector embedding generation.
image_generation: text to image or image to image.
audio_transcription: speech to text.
audio_generation: text to speech.

Vendors can add provider-specific values (anthropic.tool_use) but the core set is standardized.

Low cardinality is the design requirement

Observability backends use attributes for two things: filtering (which is fine at any cardinality) and grouping in time-series rollups (which requires low cardinality, typically tens of distinct values). High-cardinality attributes (user ids, request ids) explode storage when used for grouping.

gen_ai.operation.name is intentionally low-cardinality. Six values cover 99 percent of real traffic. This is what lets backends pre-compute per-operation aggregates cheaply.

Don't conflate with model id

The single biggest discipline mistake is folding model id into operation name: chat-gpt-5.5, chat-claude-opus-4-7. This makes the attribute high-cardinality and breaks grouping.

Model id is a separate attribute: gen_ai.request.model. Keep them separate; use both as orthogonal grouping axes when you need the cross-product.

Stable vs incubating

OTel attributes are tagged stable or incubating. Stable attributes carry compatibility guarantees across SDK versions. gen_ai.operation.name is stable as of the 2026 spec; some newer attributes (chunk-level events, multi-modal media types) are still incubating. Pin dashboards to stable attributes only.

Dashboard partitioning

Sampling policies

Cost attribution and operational practices

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

OpenTelemetry GenAI semantic conventions define gen_ai.operation.name with values like chat, text_completion, embeddings.
OpenLLMetry (Traceloop) sets the attribute automatically on every instrumented call.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat other OTel GenAI attributes do you typically pair with gen_ai.operation.name?

gen_ai.system (provider), gen_ai.request.model (model id), gen_ai.usage.input_tokens, gen_ai.usage.output_tokens. The five together give you per-provider, per-model, per-operation cost and latency analytics.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Aggregating latency across all spans without grouping by operation. Embeddings drag the average down and the chart is meaningless.

60 second bullets to scan on the way to the call.

Definition: low-cardinality enum identifying the kind of LLM operation
Canonical values: chat, text_completion, embeddings

Primary sources. Browse if you want the original framing.