When should you use token-level F1 vs exact match in a QA evaluation?

**Exact match** and token F1 are the two classic automated metrics for question answering where there is a short reference answer to compare against. They sit at opposite ends of a strictness scale. Exact match is all or nothing. The prediction must equal the reference string (after normalising case, punctuation, and articles) to score 1, otherwise it scores 0. This works well for short, canonical answers: a date like 1066, an entity like Marie Curie, or a number. There is one right string, so strictness is fair. Token F1 is gentler. It treats both the prediction and the reference as bags of tokens, then measures their overlap. You get partial credit when answers share words but differ in phrasing. 'The Battle of Hastings in 1066' overlaps heavily with 'Hastings, 1066', so F1 stays high even though exact match would score 0. The catch with F1 is that overlap is not correctness. A prediction with the right entity but the wrong date still shares tokens with the reference, so it can earn a misleadingly high score while being factually wrong. Pick exact match when the answer space is tight and a near-miss is still a miss; pick F1 when phrasing varies and partial overlap is genuinely informative.

**Exact match** scores 1 only when the normalised prediction equals the normalised reference. It is the right choice when the answer has a single canonical form: dates, numbers, entity names, multiple-choice letters. A near-miss is a real miss there, so all or nothing is honest. Token F1 treats prediction and reference as token bags and scores their overlap. It awards partial credit when the same fact is phrased differently. That suits extractive QA where answer spans vary in boundaries and wording, like SQuAD. SQuAD reports both metrics precisely because exact match alone is too brittle for free-form spans. The trap is assuming F1 is strictly better. F1 measures overlap, not truth. A prediction with the right entity but the wrong date still shares tokens with the reference. It can score 0.7 while being factually wrong. So the real decision rule is about cost. If a partially-correct answer is partially useful to your user, F1 reflects that. If a wrong fact is a hard failure regardless of overlap, exact match protects you. Most QA leaderboards report both side by side rather than choosing one.