Fill in the OpenTelemetry GenAI attributes that record input and output token counts

Fill in blank·Easy·4.0 · 0·~1 min·Asked atGong Rephrase Ai Stability Ai

Attempt it

Per the OpenTelemetry GenAI semantic conventions, the standard span attribute for prompt tokens is and the standard attribute for completion tokens is .

The two stable attributes are **gen_ai.usage.input_tokens** and **gen_ai.usage.output_tokens**. **Where they live.** Both sit under the gen_ai.usage namespace inside the OpenTelemetry GenAI semantic conventions. The conventions standardize attribute names so every observability vendor can render token counts the same way without translation. **Why input/output instead of prompt/completion.** Older provider APIs (the OpenAI Completions API in particular) used prompt_tokens and completion_tokens. That naming made sense for the original completion shape (a single prompt string, a single completion string). It does not fit chat (multiple messages, no single 'prompt') or streaming (partial output during generation) or tool calls (output is structured, not a completion). The OTel convention chose input/output deliberately for forward compatibility. Whatever shape the model serves (chat, completion, embedding, image generation), input tokens are what went in and output tokens are what came out. **Practical use.** Sum input_tokens times price per million plus output_tokens times price per million for the cost. Aggregate by model to find which model dominates spend. Aggregate by tenant_id to attribute costs per customer.

**gen_ai.usage.input_tokens** for prompt-side tokens, **gen_ai.usage.output_tokens** for completion-side tokens. Both are stable, namespaced attributes in the OpenTelemetry GenAI semantic conventions. **The naming choice.** The OTel working group deliberately chose input/output over prompt/completion. The older OpenAI Completions API used prompt_tokens and completion_tokens, but those names assume a request-response shape with one prompt and one completion. They do not fit chat (a list of messages, no single prompt), streaming (token by token output), or tool calls (structured output, not a completion). Input and output are shape-neutral. **Why standardization matters.** Without it, every provider's SDK invents its own attribute keys and every observability vendor has to write a translation layer. Anthropic might emit usage.in_tokens, OpenAI usage.prompt_tokens, Google generation.token_count.input. Dashboards built on one provider break when you swap providers. With the OTel convention, every conforming SDK writes the same names, every conforming backend reads the same names, and dashboards are portable. **The wider gen_ai namespace.** input_tokens and output_tokens are part of a larger set: gen_ai.system (the provider), gen_ai.request.model (the model id), gen_ai.request.temperature, gen_ai.response.finish_reasons, gen_ai.operation.name (chat, completion, embedding). Together they form the standard LLM span schema. Once your traces follow it, any OTel-native backend can ingest them: Langfuse, Phoenix, OpenLLMetry to anywhere, Datadog LLM Observability. **Cost calculation.** Sum input_tokens times input price plus output_tokens times output price, joined to a price table indexed by model and effective date. The OTel conventions do not include cost as an attribute; that is a backend computation derived from the token counts and the model name.

TL;DR

gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. The convention uses input/output (not prompt/completion) so chat and streaming shapes fit the same names.

Memory aid

Easy to grasp

Imagine every shipping company uses a slightly different label for 'how heavy is the box' and 'how heavy is the return package'. Customs gets a headache reading every shipment. OpenTelemetry is like a global standard that says: everyone calls the incoming weight input_tokens and the outgoing weight output_tokens, and they all live under the same gen_ai.usage shelf so any tool can find them. Once the names match, dashboards built for one provider work for another with zero changes.

Key concepts

Observability and tracing LLM system design Tokenization

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Token counts are the most important attribute on any LLM span because cost, latency proxy, and capacity planning all derive from them. The OpenTelemetry GenAI semantic conventions standardize the names so dashboards and cost calculators work across providers without translation. The two stable attributes are gen_ai.usage.input_tokens for what went in and gen_ai.usage.output_tokens for what came out.

This deep dive explains the naming choice (why input/output and not prompt/completion), the stable versus incubating distinction (why pinning dashboards to incubating keys is a future-breakage risk), how the convention handles modern shapes like chat streaming and prompt caching, and what to do when legacy SDKs emit non-standard keys.

Mental model: input_tokens and output_tokens are the universal language. Once your traces speak it, any OTel-native backend can compute cost, alert on regressions, and aggregate per-model spend with zero custom code.

The stable attribute names and namespace

The two attributes are part of a larger gen_ai semantic family.

gen_ai.usage.input_tokens

Stable. Integer. Counts the tokens that entered the model on this call. For chat, this is the tokenized concatenation of the message history (system, user, assistant, tool messages). For embedding, this is the tokenized input text. For completion, this is the prompt string.

gen_ai.usage.output_tokens

Stable. Integer. Counts the tokens the model generated. For chat and completion, this is the new content the model produced. For embedding, this is conventionally zero or omitted (the output is a vector, not text).

The wider gen_ai family

Other stable attributes on the same span include gen_ai.system (provider id like 'openai', 'anthropic'), gen_ai.request.model (model name), gen_ai.request.temperature, gen_ai.response.finish_reasons (array of stop reasons), and gen_ai.operation.name (chat, completion, embedding). Together they form the standard LLM span schema.

Namespace conventions

Dot-separated. The first segment (gen_ai) scopes everything; the second (usage, request, response) sub-scopes. Backends use this prefix to identify LLM spans and apply LLM-specific dashboards. Custom attributes outside this namespace should use your own organization prefix to avoid collisions.

Why input/output instead of prompt/completion

Stable versus incubating attributes

What you do with these attributes

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

OpenAI Python SDK with OpenLLMetry instrumentation emits gen_ai.usage.input_tokens and gen_ai.usage.output_tokens on every chat completion span.
Anthropic SDK with OpenLLMetry follows the same OTel convention, so the same dashboard works across OpenAI and Anthropic.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat is the difference between stable and incubating attributes?

Stable attributes carry a compatibility guarantee: name will not change across SDK versions. Incubating attributes can be renamed in a non-breaking SDK upgrade. Pin dashboards to stable; treat incubating as best-effort.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Writing gen_ai.usage.prompt_tokens or gen_ai.usage.completion_tokens. Those names match the older OpenAI API field names but are not the OTel standard; the stable convention uses input and output.

60 second bullets to scan on the way to the call.

The two stable attribute names and the namespace they live under
Why the convention picked input/output over prompt/completion

Primary sources. Browse if you want the original framing.