List the trace spans a context-engineering pipeline should emit per call

Attempt it

Spans to emit: (1) retrieve, fetched chunk ids, scores, latency; lets you debug recall. (2) rerank, input rank, output rank, score delta per chunk; lets you debug reranker behavior. (3) assemble, final ordered chunk list with byte and token counts per slot; lets you debug what actually went into the prompt. (4) generate, input token count, output token count, finish reason, latency; lets you debug model behavior. (5) post-process, schema validation result, repair retries used. Each span carries a stable trace_id so you can replay the exact context that produced any output. Without these, a quality regression in production is a guessing game.

Your answer

A context-engineering pipeline has several stages, and a quality regression in production usually traces back to one of them. The job of tracing is to make sure you can tell which one. The minimum useful set is five spans per call, each carrying the artifact for that stage plus its timing. The **retrieve** span records the candidate chunks pulled from the vector index, their similarity scores, and the latency of the search. This is what lets you debug recall failures. The **rerank** span records the order before and after reranking, the score per chunk, and the rerank latency. This is what lets you tell whether the reranker is helping or hurting. The **assemble** span records the final ordered list of chunks that went into the prompt, the byte and token count per slot, and the prompt template version. This is what lets you reconstruct the exact prompt later. The **generate** span records the input token count, output token count, the finish reason (stop, length, tool, content_filter), and end to end latency. The **post-process** span records schema validation results and how many repair retries it took. Every span carries the same trace_id, so an investigation starts from the bad output and walks backward through the spans until the regression is localized.

Five spans cover a production context-engineering pipeline. Each one has an artifact, a timing, and the trace_id that ties the call together. **retrieve** holds the candidate ids, scores, the query embedding hash, and the latency of the ANN lookup. It debugs recall: was the right chunk even in the candidate pool? **rerank** holds the input rank, the output rank, the rerank score per chunk, and the rerank model version. It debugs whether the reranker is doing useful work and surfaces drift when a model upgrade silently changes ranks. **assemble** is the most valuable span. It holds the final ordered chunk list, the byte and token count per slot, the role marker each piece landed in, and the prompt template version. It is the span that lets you reproduce the exact prompt offline, which is what every postmortem needs. **generate** holds the input and output token counts, the finish reason, the model id, sampling params, and the response latency. It connects an output to its model state. **post-process** holds the schema validation result and any repair retries. All five share a trace_id. When a user reports a bad answer, the operator looks up the trace_id and walks the spans in order. The regression localizes to one stage, usually within minutes. Without spans, the same investigation is a guessing game across the whole pipeline.

Quick reference

TL;DR

Emit one span per stage (retrieve, rerank, assemble, generate, post-process), each carrying the artifact and the timing, all sharing one trace_id so you can replay the exact prompt that produced any output.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine a chef in a busy kitchen and a customer who hated their meal. If the chef kept no notes, you can only ask, was it the ingredients, the prep, the cooking, the plating, or something the server did at the table? You cannot tell. Now imagine the kitchen logs each step: which tomatoes came out of the crate, how long they were prepped, what temperature the pan was, when the plate left the pass, how the server presented it. The same complaint becomes a five-minute investigation instead of a guessing game. Tracing spans do this for an LLM pipeline. Every stage writes a log, every log shares one ticket number, and a bad output is traceable end to end.

Key concepts

Context engineering Observability and tracing LLM evaluation Retrieval-augmented generation

Detailed answer & concept explanation

~4 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

4 minutes: name the five spans, explain why assemble is load-bearing, describe trace_id correlation, name one open-source and one commercial tool, describe an offline replay workflow.

Real-world examples

Real products, models, and research that use this idea.

LangSmith ships span types that map cleanly to retrieve, rerank, generate and is the default tracing layer for LangGraph and LangChain pipelines.
Arize Phoenix is an open-source tracing UI that natively understands OpenTelemetry LLM semantic conventions and surfaces per-span artifacts.
Langfuse provides per-trace cost attribution and prompt-template versioning, used by mid-sized RAG teams in 2026.
Anthropic's own internal evaluation tooling for Claude Opus 4.7 retains assembled prompts and outputs for offline replay against newer model checkpoints.
OpenTelemetry GenAI semantic conventions standardize span attributes for gen_ai.system, gen_ai.request.model, and prompt_tokens; widely adopted in production stacks.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you redact PII from spans without losing reproducibility?

Sign in or go Pro to see the approach for this follow-up.

QWhat span schema lets you debug a regression after swapping rerankers?

Sign in or go Pro to see the approach for this follow-up.

QHow would you tie LLM-as-judge eval scores back to the producing pipeline?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Logging only the final prompt and the final answer. You can see the failure but you cannot tell which stage caused it; retrieve, rerank, and assemble each have to emit their own span.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.