Why might F1 score be a better choice than raw accuracy for evaluating entity extraction?

Attempt it

Quick reference

TL;DR

F1 is the harmonic mean of precision and recall, making it sensitive to both false positives and false negatives. Accuracy misleads when classes are imbalanced.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine you are looking for gold nuggets in a river. Accuracy asks 'what percentage of all the rocks and gold did you classify correctly?' If you just say everything is a rock, you are right 99% of the time because gold is rare. Great accuracy, but useless. F1 asks two questions at once. First, when you say something is gold, how often are you right? That is precision. Second, of all the gold that was actually there, how much did you find? That is recall. F1 is the combined score that forces you to be good at both. If you never find gold (zero recall), your F1 is zero, even though your accuracy was 99%. In entity extraction, entities are the gold nuggets: rare and important. Accuracy hides your failure to find them. F1 exposes it.

Key concepts

F1 Score Precision Recall Harmonic Mean Class Imbalance

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

5 min: define F1 with the formula, explain precision and recall, demonstrate the class imbalance problem with a concrete example, show why accuracy misleads, and close with the diagnostic workflow (low F1 -> check precision vs recall separately).

Real-world examples

Real products, models, and research that use this idea.

SpaCy's named entity recognition models are evaluated using entity-level F1 on the OntoNotes benchmark, where raw accuracy would be meaningless due to the dominance of non-entity tokens.
Medical NER systems that extract drug names and disease mentions from clinical notes report F1 because missing a drug mention (low recall) can have patient safety consequences.
Stripe uses F1 to evaluate entity extraction in its fraud detection pipeline, where catching fraudulent entities (recall) is critical but false alarms (precision) create operational costs.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat is the F-beta score and when would you use it instead of F1?

Sign in or go Pro to see the approach for this follow-up.

QHow does entity-level (span-level) F1 differ from token-level F1 in NER?

Sign in or go Pro to see the approach for this follow-up.

QWhen two models have the same F1 but different precision/recall breakdowns, how do you choose?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating accuracy as a reliable metric for entity extraction. When entities are rare, a model that predicts nothing can achieve very high accuracy while being completely useless.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.

Van Rijsbergen, Information Retrieval (Chapter on Evaluation)