Trace at the top, observations in the middle (span, generation, event), scores attached to the side. Three layers, not four.
Think of a school report. The whole report is the trace, covering one term. Each subject inside the report is an observation: math, history, art. Subjects come in flavors too. Some have homework grades, some have exam grades, some have one-off events like a field trip. Scores are the teacher comments stapled on the side of each subject or on the cover of the whole report. The model is three layers because that mirrors how a real LLM request happens: one customer interaction at the top, several units of work inside, and quality judgments dangling off the side without being part of the work itself.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Langfuse's UI looks like any other tracing tool at first glance: traces expand into nested spans, the spans show inputs and outputs, the dashboards summarize the lot. The underlying data model is more opinionated than that, and the opinions are what make Langfuse useful for LLM workloads in ways that pure OTel is not.
This deep dive walks through the three layers, the observation kinds inside the middle layer, and the structural side-channels (scores, sessions, datasets, prompts) that sit beside the tree. Knowing where each piece lives is what keeps the dashboards, evals, and cost reports populating correctly.
Layer one: trace
A trace represents one end to end execution. The trace is the unit you reason about when answering "what did the system do for this user request?" One trace per user message, one trace per scheduled job run, one trace per API call to your service.
Trace fields
The trace carries name, user_id, session_id, release tag, version tag, tags, public flag, and a metadata bag. Cost, latency, and total token usage are computed as rollups from contained observations and exposed on the trace for filtering and dashboards.
What is not a trace
A multi-turn chat is not one trace. Each turn is one end to end execution and gets its own trace; the turns share a session_id. A scheduled eval run that processes 500 dataset items is not one trace; each item is one trace, and they share a dataset_run_id.
This discipline matters because cost and quality are per-execution numbers. Megatraces flatten that granularity and make the dashboards useless.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Layer | What it represents | Nests under | Typical example |
|---|---|---|---|
| Trace | One end to end request | Nothing (top level) | One user message through the full stack |
| Span | A non-LLM unit of work | Trace or another observation | Vector retrieval call |
| Generation | An LLM call with cost-bearing fields | Trace or another observation | OpenAI chat completion |
| Event | A point in time marker, zero duration | Trace or another observation | Cache hit, guardrail trigger |
| Score | An eval result attached to a trace or observation | Side-attached, not a tree node | User thumbs-down, LLM-judge faithfulness=0.6 |
Real products, models, and research that use this idea.
- Langfuse's quickstart instrumentation guide demonstrates exactly this hierarchy: `langfuse.trace(...).generation(...).score(...)` in three nested SDK calls.
- ChatGPT-like products typically open one trace per user message, contain multiple generations (router, retrieval, synthesis), and emit a `user_feedback` score from the frontend on thumbs click.
What an interviewer would ask next. Try answering before peeking at the approach.
QIf a multi-turn chat has 8 turns, do you model it as 1 trace with 8 generations or 8 traces with a shared session_id?
Eight traces with a shared session_id. Each turn is a complete end to end execution; cost and judge attribution are per-turn. The session_id index keeps the chat together for navigation; the per-turn shape preserves the granularity the dashboards need.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating scores as a fourth layer in the tree. They are attachments, not nodes; a score has a parent observation or trace but no children.
60 second bullets to scan on the way to the call.
The three layers from top to bottom
Three kinds an observation can be and what differentiates them
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.