A trajectory is the full sequence of thought, action, observation, thought, action steps the agent took to reach its final answer. Final-answer accuracy alone misses whether the path was efficient, correct, or safe. Trajectory evaluation scores intermediate steps too: were the right tools called, in the right order, with the right arguments.
**A trajectory** is the full record of everything an agent did during a run. Every thought the model wrote out, every tool call it made, every observation it received back, in the order they happened. If you replayed the trajectory step by step, you would see exactly how the agent went from the user's task to its final answer.
The trajectory matters because the final answer alone hides too much. Two agents can both produce the correct answer through completely different paths. One might solve the task in three clean steps; the other might call the wrong tool five times, recover by accident, and stumble onto the right answer after fifteen steps. Final-answer accuracy says they are equally good. Trajectory evaluation says they are not.
Production agent evaluation looks at trajectories on three axes. Did the agent call the right tools in the right order? Did it call them with sensible arguments? Did it avoid unnecessary steps and dangerous side effects? Frameworks like Langfuse, LangSmith, Arize Phoenix, and Weave capture trajectories as structured trace data, and benchmarks like SWE-bench, AgentBench, and TAU-bench score them.
The short version is: final-answer eval tells you whether the agent got there. Trajectory eval tells you whether it deserves to.
**A trajectory** is the full sequence of thought, action, observation, thought, action, observation steps an agent went through on a single run. It is the agent's equivalent of a profiler trace or a debugger session: every model output, every tool call, every observation, in order, with timestamps and metadata.
Trajectory evaluation matters because final-answer accuracy alone misses everything that matters operationally. An agent that lands the right answer in fifteen wandering steps is fragile: on the next slightly different task it will probably wander differently and fail. An agent that lands the right answer in three clean steps is robust: its reasoning was structured and its tool use was efficient. Final accuracy reports them as equally good. Trajectory scoring reports them honestly.
The canonical evaluation dimensions are four. **Tool-selection correctness**: did the agent call the right tool at each step, given what it knew. **Argument correctness**: were the arguments well-formed and sensible. **Path efficiency**: how close to the minimum step count did the agent run. **Safety and side-effect correctness**: did the agent avoid state-changing tools it should not have used, and did its retries respect idempotency.
Frameworks like Langfuse, LangSmith, Arize Phoenix, Helicone, and Weave capture trajectories as nested trace spans you can inspect. Benchmarks like SWE-bench (real GitHub issues), AgentBench (multi-environment), GAIA (assistant tasks), and TAU-bench (tool-use focused) score both final answers and trajectories. LLM-as-judge graders also work well at trajectory scale, since a human cannot review every step of every run at production scale.
Quick reference
TL;DR
A trajectory is the full thought, action, observation sequence an agent ran through. Trajectory evaluation scores the path, not just the final answer, so you can catch inefficient, unsafe, or accidentally correct runs.
Picture two students who both turn in the same correct math answer. One solved it cleanly in three lines. The other wrote two pages of crossed-out attempts, accidentally divided by zero twice, and somehow landed on the right number. As a teacher, do you grade them the same? Probably not. The final answer hides everything you actually care about: did they understand the method, did they avoid dangerous mistakes, would they get the next problem right too. A trajectory is the agent's two pages of work. It is every thought, every tool call, every observation in order. Looking at the trajectory tells you whether the agent solved the task for the right reasons or got there by luck.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
One-shot completions where there is no loop and no tool sequence. The 'trajectory' collapses to a single prompt and response; standard prompt evaluation methods apply directly.
Early prototyping where you only need to confirm the agent can produce any reasonable answer. Heavyweight trajectory analysis is overkill before the basic loop even works; defer until you have a passing baseline.
Privacy-sensitive deployments where trajectory data contains user PII. Trace capture creates an audit surface; ensure scrubbing, retention, and access policies are in place before turning it on broadly.
Quick scan · 2–4 min
Everything important, quickly.
5 min: define trajectory as the ordered thought, action, observation sequence, explain why final-answer eval misses path-level information, list the four canonical scoring axes, name two or three observability tools and two or three benchmarks, and mention LLM-as-judge as the scaling mechanism.
Real-world examples
Real products, models, and research that use this idea.
1Langfuse, LangSmith, Arize Phoenix, Helicone, and Weave all capture agent trajectories as nested trace spans; each turn becomes an inspectable call stack with model, tool, and observation spans.
2SWE-bench scores real GitHub issue resolutions but exposes the full trajectory so researchers can debug why a model failed on specific repos.
3TAU-bench (Tool-Use Agent Benchmark) is explicitly trajectory-focused; tool-call sequence quality, not just final answer, contributes to the score.
4AgentBench evaluates trajectories across eight environments (web, code, OS, etc.), exposing per-step latency, cost, and success metrics for cross-task comparison.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you score thousands of trajectories without reading each one by hand?
Sign in or go Pro to see the approach for this follow-up.
QWhat does a trajectory tell you that you cannot see in the final answer alone?
Sign in or go Pro to see the approach for this follow-up.
QWhy do production agent teams emit trajectories in OpenTelemetry format?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Grading agents on final-answer accuracy only. Two agents can both land the right answer with wildly different paths; the wasteful or unsafe one will fail on the next slightly different task.