Exact match is all or nothing and unambiguous for canonical-form answers; token F1 gives partial credit for phrasing variation but can also reward hallucinated spans that happen to overlap.
Imagine grading a one-line answer on a quiz. With exact match you are a strict teacher: the answer matches the key word for word, or it gets a zero. That is fair when the answer is a date or a name, because there is one right way to write it. Token F1 is a lenient teacher who gives partial credit for getting some of the words right. That helps when students phrase the same fact differently. But the lenient teacher has a blind spot: a student who writes the right name with the wrong date still earns points for the matching words, even though the answer is factually wrong. So strict grading is safe for short, fixed answers, and lenient grading is useful for varied phrasing, as long as you remember it can reward near-misses that are actually wrong.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Exact match and token-level F1 are the two foundational automated metrics for question answering against a short reference answer. They were popularised together by extractive QA benchmarks like SQuAD, and the choice between them is one of the most common practical decisions in eval design. They are cheap, deterministic, and require no judge model, which is exactly why they remain the first line of automated grading even in an era dominated by LLM judges.
The two metrics sit at opposite ends of a strictness scale. Exact match demands a perfect normalised string and gives nothing for being close. Token F1 measures overlap and hands out partial credit. Neither is universally correct, and choosing between them is really a statement about how much a near-miss should cost in your specific task.
The deep dive walks what each metric actually computes, why F1 decomposes into precision and recall, the precise conditions under which each is the right tool, and the failure mode that catches teams off guard: token F1 rewarding overlap that is not correctness. It closes on what replaces both once answers stop being short canonical strings.
What exact match actually computes
Exact match is the strictest possible string metric. After a normalisation step, the predicted answer must equal the reference exactly to score 1, otherwise it scores 0. There is no middle ground, which is both its strength and its limitation.
The normalisation step is not optional and is where naive implementations go wrong. The standard SQuAD normaliser lowercases the text, strips punctuation, removes articles (a, an, the), and collapses whitespace. Without it, 'The Battle of Hastings' and 'battle of hastings' would score 0 against each other despite being identical answers. Skipping normalisation is one of the most common bugs in homegrown eval harnesses, and it produces a quiet, systematic underscoring that looks like a model regression when it is really a string-handling bug.
When multiple gold answers exist, exact match takes the maximum over them: a prediction scores 1 if it matches any acceptable reference. This is how datasets encode legitimate variation (for example, '1066' and 'the year 1066') without abandoning strictness. The metric stays binary per example; the variation lives in the reference set, not in the scoring rule.
Exact match shines when the answer space is tight and canonical: a year, a quantity, a person's name, a yes or no, a multiple-choice letter. In those cases a near-miss really is a miss, so all or nothing scoring is honest rather than harsh. Its weakness is brittleness on free-form spans, where two equally valid phrasings of the same fact each score 0 against the other, and where the metric punishes correct content for arriving in an unexpected surface form.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- The SQuAD benchmark reports exact match and token F1 side by side as the standard extractive QA leaderboard pair.
- Natural Questions and TriviaQA use normalised exact match plus token F1 for short-answer evaluation.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does SQuAD report both exact match and token F1 rather than picking one?
Exact match underrates correct but reworded spans; F1 captures partial overlap but is noisier. Reporting both shows brittleness and partial credit together, so a system strong on one but weak on the other is visible rather than hidden behind a single number.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating token F1 as strictly better than exact match. F1 rewards overlap, so an answer with the right entity but wrong date can score high while being factually wrong.
60 second bullets to scan on the way to the call.
What exact match scores and when its strictness is fair
How token F1 decomposes into precision and recall
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.