When you evaluate an agent, the easy thing to check is the **final answer**. Did it return the right number, the right summary, the right fix? That is final answer accuracy, and it only looks at the endpoint.
The problem is that the endpoint hides how the agent got there. An agent can stumble onto the right answer through luck, take thirty tool calls when five would do, or pick the wrong tool and then accidentally recover. Final answer accuracy gives all of these full marks.
Trajectory evaluation grades the whole path instead. It looks at the sequence of steps the agent took: which tools it called, with what arguments, in what order, and whether each step followed sensibly from the last observation.
Three things matter in a trajectory. Efficiency, did it use a reasonable number of steps. Tool use correctness, did it call the right tools with the right inputs. Plan coherence, did each step make sense given what came before.
The key idea is that a correct answer from a broken process will not survive the next input. Trajectory evaluation catches that fragility before it ships.
**Trajectory evaluation** scores the agent's path, not just its endpoint. Final answer accuracy asks only one question. Is the last output correct?
That misses a lot. An agent can reach the right answer by luck, by a wasteful thirty step detour, or by calling the wrong tool and accidentally recovering. All three score perfectly on endpoint accuracy. None is a process you would trust on the next input.
Trajectory evaluation grades three properties of the step sequence. Efficiency is step count versus the minimum needed, plus redundant tool calls. Tool use correctness is the right tool with the right arguments in the right order. Plan coherence asks whether each step follows from the previous observation, with sensible recovery after errors.
The distinction matters because agents are sequential decision makers, not single shot predictors. A lucky correct answer is not repeatable. A wasteful trajectory burns tokens and latency even when correct. An incoherent trajectory signals the agent does not understand the task. It will fail on a harder variant. Benchmarks like tau-bench and AgentBench report path-level signals because endpoint accuracy alone overstates how reliable an agent really is.
Quick reference
TL;DR
Trajectory evaluation scores the path (tool choice, step count, plan coherence, recovery), not just the endpoint, so it catches lucky right answers and wasteful or incoherent routes.
Imagine grading a student's maths homework. Final answer accuracy only checks the boxed number at the bottom. If it matches the key, full marks, even if the student guessed or copied a friend.
Trajectory evaluation grades the working. It looks at each line: did the student pick the right method, skip pointless steps, and recover sensibly after a mistake? Two students can both write the correct final number. One got there in three clean steps. The other scribbled twenty messy lines and stumbled onto it by luck.
Final answer grading gives both the same score. Trajectory grading sees that only one of them actually understands the problem and will get the next one right too. For an agent, the working is its sequence of tool calls and reasoning steps.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
A single shot task with no tool calls and one deterministic correct output. There is no trajectory to score, so endpoint accuracy already captures quality fully.
Early prototyping where you only need a rough pass or fail signal fast. Building gold traces and rubrics is expensive, and outcome accuracy is enough to triage.
Tasks with a huge space of equally valid paths and no safety invariants. Trajectory scoring over-penalises legitimate divergence and adds noise without insight.
Production monitoring where you lack per-step tracing infrastructure. Trajectory metrics need a trace span per step, so without them only the endpoint is observable.
Quick scan · 2–4 min
Everything important, quickly.
Define final answer accuracy and its blind spot, then name the four trajectory signal families, explain why outcome and process decorrelate, contrast reference matching with rubric judging, and close with the non uniqueness problem and when a strict path check is actually justified.
Real-world examples
Real products, models, and research that use this idea.
1tau-bench scores customer service agents on both task success and adherence to the correct tool sequence, exposing agents that pass on outcome but violate policy on the path.
2SWE-bench measures whether a coding agent's final patch passes the hidden test suite, while SWE-bench Verified and trace tooling let teams inspect the edit test trajectory behind each pass.
3AgentBench reports per environment success, but its traces reveal step count and tool misuse patterns that separate a frontier model like Claude Opus 4.7 from a weaker one even at similar success rates.
4LangSmith and Langfuse capture per-step spans so teams can run rubric-based LLM-as-judge over the full tool-call trajectory, not just the final agent message.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you score a trajectory when many valid step sequences solve the same task?
Sign in or go Pro to see the approach for this follow-up.
QHow do you combine an outcome reward with a process signal without one drowning the other?
Sign in or go Pro to see the approach for this follow-up.
QWhat trajectory signals best predict that an agent will generalise to harder unseen tasks?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Assuming a correct final answer means a correct process. An agent can reach the right output through a lucky recovery or a wasteful path that breaks on the next input.