Generation is the LLM-call shape with first-class fields for model, tokens, and cost. Plain span is for everything else inside the trace.
Picture a hardware store with two kinds of receipts. The general receipt has blank lines you fill in by hand. The lumber receipt already has columns for board feet, species, and grade. If you buy nails you use the general one. If you buy lumber you use the lumber one because the columns are already there and the system can total board feet at the end of the month. A Langfuse generation is the lumber receipt for LLM calls. The columns for tokens, model, and cost are already there. A plain span is the general receipt. Use the wrong one for lumber and you get a working pile of receipts that the month-end totals cannot read.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Langfuse's observation model splits the world into three kinds, and the split exists for a reason: LLM calls have enough structural commonality that promoting them to a first-class shape lights up an entire dashboard and eval surface that a freeform span cannot provide. Teams that miss this distinction ship traces that look complete on screen and produce empty cost dashboards in the back office.
This deep dive walks through what generation actually adds beyond span, what work each downstream feature does on top of that structure, and how to keep the convention consistent across hand-written instrumentation, OpenInference auto-instrumentation, and OpenLLMetry exporters.
The three observation kinds
A Langfuse trace contains observations. There are three observation kinds and the choice is structural, not stylistic.
Span
A generic unit of work with a start time, an end time, a name, an input field, an output field, and a metadata bag. Use for retrieval, reranking, tool execution, validation, parsing, post-processing, and any non-LLM step. The metadata bag is freeform JSON.
Generation
A specialized span representing an LLM call. Adds first-class fields for model, modelParameters (temperature, max_tokens, top_p, etc.), structured input as a list of messages, structured output, and a usage object containing input tokens, output tokens, and total tokens. Langfuse joins model plus usage against its model price table to compute cost automatically.
Event
A point in time marker. No duration. Use for things like cache_hit, guardrail_triggered, rate_limit_exceeded, prompt_template_loaded. Events are cheap and queryable, which makes them ideal for instrumenting branch decisions.
All three nest under the trace. Spans and generations can nest under each other freely; events attach to whichever observation was active when they fired.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
from langfuse import Langfuse
langfuse = Langfuse()
trace = langfuse.trace(name='chat_request', user_id=user.id)
# Retrieval is non-LLM work: plain span
with trace.span(name='retrieve', input={'query': q}) as s:
chunks = vector_db.search(q, k=5)
s.update(output={'chunks': len(chunks)})
# The LLM call itself: generation with cost-bearing fields
gen = trace.generation(
name='answer_synthesis',
model='gpt-5.5-mini',
model_parameters={'temperature': 0.2, 'max_tokens': 600},
input=messages,
)
resp = openai.chat.completions.create(model='gpt-5.5-mini', messages=messages)
gen.end(
output=resp.choices[0].message.content,
usage={
'input': resp.usage.prompt_tokens,
'output': resp.usage.completion_tokens,
'total': resp.usage.total_tokens,
},
)Real products, models, and research that use this idea.
- Langfuse's own quickstart shows `langfuse.generation(name='openai-completion', model='gpt-5.5', input=msgs, output=resp, usage={...})` as the canonical LLM call wrapper.
- OpenLLMetry's OpenAI auto-instrumentation emits OpenInference `llm` spans that the Langfuse OTel exporter rewrites as generations during ingest.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you handle a single conversational turn that calls the model three times (retrieval, rerank with LLM, synthesis)?
Three generations under one trace. The rerank-with-LLM is itself a generation, not a span, because it consumes tokens and produces an output; the retrieval that fed it is a span. The model-mix chart will then show three calls per turn, which is what you want.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Logging an LLM call as a plain span with token counts stuffed into the metadata bag. The cost dashboard then shows zero spend and you cannot tell why.
60 second bullets to scan on the way to the call.
Which Langfuse observation kind carries first-class model, tokens, and cost fields
Why a plain span with metadata cost cannot join the price table
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.