Explain what OpenInference adds on top of plain OpenTelemetry for LLM tracing

Flashcard·Medium·4.0 · 0·~30s·Asked atAutodesk Lakera Tesla

Attempt it

OpenInference is the open semantic-convention layer Arize maintains for LLM and agent telemetry. It builds on OpenTelemetry by adding richer attribute conventions for LLM-specific concepts: chain steps, retrieval results with per-chunk scores, reranker spans, tool calls, embeddings inputs, and agent step kinds. It pre-dated parts of the OTel GenAI conventions and overlaps with them; the OTel spec has gradually adopted similar ideas. Tracing libraries that emit OpenInference can be read by Phoenix and by any OTel backend that understands the attribute keys.

Your answer

OpenTelemetry has two parts: the wire protocol (how spans move on the network) and the semantic conventions (what attribute names mean). The protocol is universal. The conventions are domain-specific. There are conventions for HTTP, databases, messaging, and more. **OpenInference is a semantic-convention layer for LLM workloads.** Arize created it in 2023 to standardize attribute names for LLM-specific concepts that OTel's general semantic conventions did not yet cover. OpenInference adds attribute conventions for: - **Span kinds** like `llm`, `chain`, `retriever`, `reranker`, `tool`, `embedding`, `agent` - **Retrieval results** with per-document attributes (id, content, score, metadata) - **Tool calls** with structured arguments and outputs - **Embeddings** with input text and resulting vector dimensions - **Agent steps** with step kind and intermediate reasoning The wire protocol underneath is still OTel. A span emitted with OpenInference attributes is a normal OTel span; any OTel-compatible backend can ingest it. Phoenix, the open-source observability tool also from Arize, understands OpenInference natively and renders the LLM-specific shapes directly. **The OTel GenAI conventions came later** and overlap with OpenInference. The two specs are converging; production stacks use whichever their tooling supports.

**OpenInference is a vocabulary, not a wire format.** That distinction is what makes it interoperable. The OTel protocol carries spans; the semantic conventions define attribute names; OpenInference defines LLM-specific attribute names that sit alongside OTel's general ones. **What it adds.** Plain OTel had `http.method` and `db.statement` but had no idiomatic way to express "this span was a vector retrieval that returned five documents with similarity scores 0.91, 0.87, ...". OpenInference adds that vocabulary: `openinference.span.kind=retriever`, `retrieval.documents.0.document.id`, `retrieval.documents.0.document.score`, `retrieval.documents.0.document.content`, and so on through the result list. The same expansion exists for chains, rerankers, tool calls, embeddings, and agent steps. Each gets a span kind value plus a set of attribute conventions that describe its inputs and outputs. **Why it pre-existed OTel GenAI.** Arize shipped OpenInference in 2023 because they needed conventions for Phoenix and OTel's GenAI semantic conventions did not exist yet. OTel GenAI began rolling out in 2024 and continues to be refined. The two specs overlap heavily today; OpenInference covered chain, retriever, reranker, agent earlier and more thoroughly, OTel GenAI covered the basic LLM-call shape first. **Practical interop.** A Phoenix-instrumented app emits OTel spans with OpenInference attributes. Phoenix renders them as LLM-aware traces. Langfuse, with its OTel ingest, also understands the attributes and re-renders them as generations and spans. Pure-OTel backends (Tempo, Honeycomb, Datadog) see them as standard spans with unfamiliar but harmless attribute keys; basic dashboards work, LLM-specific views require attribute filters.

TL;DR

Arize's open semantic-convention layer for LLM and agent traces. Adds chain, retriever, reranker, tool, and agent attributes on top of OTel. Phoenix consumes it natively.

Memory aid

Easy to grasp

Imagine OpenTelemetry as a universal alphabet that says "here is how you spell words." Useful, but it does not tell you how to write a recipe. OpenInference is a recipe-specific extension that says "if you are writing a recipe, here is how to label the ingredient list, the steps, and the cooking time." It is still the same alphabet underneath. Anyone who reads recipes (Phoenix) understands the labels natively. Anyone who reads any text (other OTel backends) can still read the recipe because the alphabet has not changed; they just see the labels as generic words.

Key concepts

Observability and Tracing RAG Agents

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

OpenInference exists because Arize shipped Phoenix in 2023 and needed a way to describe LLM traces that plain OTel could not yet express. Plain OTel had HTTP and database conventions but no idiomatic way to encode "this span was a retrieval step that returned five documents with similarity scores." Arize wrote that vocabulary, called it OpenInference, and released it as an open spec.

OTel's own GenAI semantic conventions began materializing in 2024. The two specs overlap. This deep dive walks through what OpenInference contributes structurally, how it relates to OTel GenAI in 2026, and the practical patterns for keeping both worlds usable as the specs converge.

Convention layer, not protocol

OpenTelemetry has two distinct layers that are easy to conflate.

The protocol layer

OTLP, the wire format. It defines how spans, metrics, and logs serialize and travel over the network. Every OTel SDK and every OTel-compatible backend speaks OTLP. There is one wire format.

The semantic-conventions layer

Namespaces and attribute names with documented meanings. OTel has conventions for HTTP (http.method, http.status_code), databases (db.statement, db.system), messaging (messaging.system, messaging.destination), and now GenAI (gen_ai.request.model, gen_ai.usage.input_tokens).

The conventions layer is what makes spans portable across teams and tools. If everyone agrees that http.status_code means the HTTP status code, every backend can filter on it.

Where OpenInference fits

OpenInference is a semantic-conventions layer for LLM-specific concepts. It sits at the same architectural level as the OTel HTTP or DB conventions. The wire is still OTLP; the spans are still OTel spans; the attributes follow OpenInference's naming.

This is why a backend like Langfuse that ingests OTel can read OpenInference-instrumented apps. The wire is what it understands; the attribute names are what it parses out of the standard span attribute bag.

What OpenInference adds

Why it pre-existed OTel GenAI

Interop patterns in 2026

OpenInference versus OpenLLMetry

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Concern	Plain OTel	OpenInference	OTel GenAI
LLM call basics (model, tokens)	Custom keys required	llm.model_name, llm.token_count.*	gen_ai.request.model, gen_ai.usage.*
Prompt content	No standard	Attributes (older), events (newer)	Events (gen_ai.user.message, etc.)
Retrieval results	No standard	retrieval.documents.* indexed array	Limited coverage as of 2026
Reranker spans	No standard	openinference.span.kind=reranker	Limited coverage
Agent steps	No standard	openinference.span.kind=agent + step kinds	Partial coverage, evolving
Tool calls	No standard	tool.name, tool.parameters, tool.output	gen_ai.tool.* (newer)

Real products, models, and research that use this idea.

Phoenix's auto-instrumentation for LangChain, LlamaIndex, and DSPy emits OpenInference-conformant spans by default, which is what powers Phoenix's RAG and agent-trace UIs.
OpenLLMetry's OpenAI auto-instrumentation emits OpenInference attributes; production stacks shipping to both Phoenix and Langfuse work with no code change.

What an interviewer would ask next. Try answering before peeking at the approach.

QIf you instrument with both OpenInference and OTel GenAI attributes simultaneously, do you double the span size?

Yes, by a small amount. The model id, token counts, and finish reason live under both naming schemes (llm.model_name and gen_ai.request.model, for example). In practice the dual-emit cost is under 10 percent of span size, and it lets you use either backend without changing instrumentation. Normalize at ingest if the cost matters.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating OpenInference as a competitor to OpenTelemetry. It is a semantic-convention layer on top of OTel, not a separate protocol.

60 second bullets to scan on the way to the call.

Why OpenInference is a semantic-convention layer rather than a wire protocol
What span kinds OpenInference adds beyond plain OTel

Primary sources. Browse if you want the original framing.