What does FActScore measure and how does it decompose long-form factual evaluation?
FActScore breaks a long generation into atomic facts, verifies each against a knowledge source, and scores the fraction supported. It turns vague long-form factuality into a precise, per-claim number.
Imagine a friend tells you a long story about a famous person. Instead of grading the whole story with one gut-feel thumbs up or down, you write down every separate claim it makes: born in this city, won that award, married this person. Then you check each claim, one at a time, against a trusted encyclopedia. Your final grade is simple: out of all those little claims, what fraction turned out to be true? That is FActScore. It does not care whether the story sounds confident or reads beautifully. It only counts how many of the small, checkable facts actually hold up against the reference. A polished paragraph with three wrong claims scores worse than a plain one where everything checks out.
Detailed answer & concept explanation~8 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
5 min: the decompose, verify, and aggregate pipeline, atomic-fact granularity, precision versus recall, knowledge-source coverage limits, retrieval versus factual error separation, and the estimated LLM verifier variant.
Real products, models, and research that use this idea.
- FActScore is the standard long-form factuality benchmark, used to score biography generation from frontier models like GPT-5.5 and Claude Opus 4.7.
- RAGAS faithfulness applies the same claim-decomposition idea, checking each extracted claim against retrieved context rather than Wikipedia.
- SAFE, the search-augmented factuality evaluator from the LongFact work, extends FActScore by sending each atomic fact to a live search engine.
- Hallucination dashboards in observability tools like Langfuse and Arize Phoenix surface per-claim support rates inspired by atomic-fact scoring.
- VeriScore and FActScore-style estimators run an LLM verifier per atomic fact to scale evaluation across thousands of generations per eval pass.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you adapt FActScore from a static Wikipedia reference to a live retrieval setting like RAG?
QWhy does FActScore measure precision rather than recall, and how would you add a recall signal?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Treating FActScore as a recall metric. It measures precision, the fraction of stated facts that are supported, and says nothing about which true facts the generation left out.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.