Why might F1 score be a better choice than raw accuracy for evaluating entity extraction?
F1 is the harmonic mean of precision and recall, making it sensitive to both false positives and false negatives. Accuracy misleads when classes are imbalanced.
Imagine you are looking for gold nuggets in a river. Accuracy asks 'what percentage of all the rocks and gold did you classify correctly?' If you just say everything is a rock, you are right 99% of the time because gold is rare. Great accuracy, but useless. F1 asks two questions at once. First, when you say something is gold, how often are you right? That is precision. Second, of all the gold that was actually there, how much did you find? That is recall. F1 is the combined score that forces you to be good at both. If you never find gold (zero recall), your F1 is zero, even though your accuracy was 99%. In entity extraction, entities are the gold nuggets: rare and important. Accuracy hides your failure to find them. F1 exposes it.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
F1 score is one of the most fundamental metrics in classification and extraction tasks, and the question of when to use it instead of accuracy is a reliable interview topic. The core insight is about class imbalance: accuracy can be deceptively high when one class dominates the data.
This deep dive covers the F1 formula, the precision/recall decomposition, why accuracy fails on imbalanced data, and practical considerations for entity extraction evaluation.
The F1 formula and what it captures
F1 is the harmonic mean of precision and recall.
Precision measures the fraction of positive predictions that are correct. If the model predicts 100 entities and 80 are actually entities, precision is 0.80. The remaining 20 are false positives.
Recall measures the fraction of actual positives that the model found. If the test set contains 200 entities and the model found 80, recall is 0.40. The remaining 120 are false negatives.
F1 combines both into one score. The harmonic mean is stricter than the arithmetic mean: it penalizes imbalance between the two components. If precision is 0.80 and recall is 0.40, the arithmetic mean would be 0.60, but F1 is 0.53. The harmonic mean pulls the score toward the lower component, ensuring that a model cannot score well by excelling at one dimension while failing at the other.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- SpaCy's named entity recognition models are evaluated using entity-level F1 on the OntoNotes benchmark, where raw accuracy would be meaningless due to the dominance of non-entity tokens.
- Medical NER systems that extract drug names and disease mentions from clinical notes report F1 because missing a drug mention (low recall) can have patient safety consequences.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is the F-beta score and when would you use it instead of F1?
F-beta generalizes F1 by adding a parameter beta that controls the precision/recall tradeoff. F1 sets beta = 1 (equal weight). F2 (beta = 2) weights recall twice as much. F0.5 (beta = 0.5) weights precision twice as much. Use F2 when missing a positive is more costly than a false alarm.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating accuracy as a reliable metric for entity extraction. When entities are rare, a model that predicts nothing can achieve very high accuracy while being completely useless.
60 second bullets to scan on the way to the call.
Define F1 as the harmonic mean of precision and recall
Write out the F1 formula
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.