Which events must every agent execution trace capture to enable debugging of production failures?
A useful agent trace records per step: input context, the model output with its tool call, the tool inputs and result or error, plus latency and token cost.
Imagine you hire an assistant to plan a trip, and they make twenty phone calls to book flights and hotels. Later the trip falls apart. To find out what went wrong, you need a recording of each call: who they called, exactly what they asked, what answer they got, and how long each call took. If all you have is the final ruined itinerary, you cannot tell which call caused the mess. An agent trace is that recording. For every step the agent takes, it saves what the agent saw, what it decided to do, which tool it called and with what inputs, what the tool returned, and the time and money that step burned. With that recording you can replay the whole run and find the exact step where things broke.
Detailed answer & concept explanation~8 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
List the six canonical per-step fields, frame the run as a distributed trace with one span per loop iteration, explain why tool errors and per step cost matter, dismiss the seed and browser distractors, and close by naming a 2026 tool such as LangSmith, Langfuse, Arize Phoenix, or OpenTelemetry GenAI.
| Field | Capture? | Why |
|---|---|---|
| Input context snapshot | Yes | Shows what the model saw that turn |
| Model response plus tool call | Yes | Shows what the model decided |
| Tool inputs and output or error | Yes | Shows what the tool actually did |
| Step latency and token cost | Yes | Enables performance and cost attribution |
| Operating system and browser | No | Irrelevant to agent reasoning |
| Sampling seed | No | Rarely exposed by the API, not a standard field |
Real products, models, and research that use this idea.
- LangSmith renders each agent run as a nested tree of spans, one per LLM call and tool call, with token cost and latency on every node, so you can click into the exact failing step.
- Langfuse models a run as a trace containing observations for each generation and tool call, capturing inputs, outputs, model, and cost for replay and evaluation.
- Arize Phoenix ingests OpenTelemetry GenAI spans and lets you inspect the prompt, tool call arguments, and tool result at each step of an agent trajectory.
- OpenTelemetry's GenAI semantic conventions standardise span attributes for model name, token usage, and tool calls, so traces from Claude Opus 4.7 or GPT-5.5 agents are portable across backends.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you keep traces useful when a single tool returns a 50 kilobyte response on every one of forty steps?
QYour agent occasionally leaks customer PII into tool inputs. How do you trace without storing the PII?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Logging only the final answer. When a 20-step run fails, the final output tells you nothing about which step broke. You need a per-step span with inputs, outputs, and tool results.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.