Describe how FActScore evaluates the factual accuracy of long-form LLM outputs. What is the atomic fact decomposition approach, how is each fact verified, and what does the final score represent?
Free · 2 AI evals / day
**FActScore** is a way to measure how factually accurate a long piece of model-generated text is, like a biography or a multi-paragraph answer. The problem it solves: you cannot just string-match a long passage against a gold answer, and asking a judge 'is this accurate?' gives one fuzzy number that hides where the errors are.
It works in three steps. First, decompose: break the text into atomic facts, the smallest claims you can check on their own. The sentence 'Marie Curie was born in Warsaw in 1867' splits into two facts: she was born in Warsaw, and she was born in 1867.
Second, verify: for each atomic fact, look up relevant context in a trusted knowledge source like Wikipedia, then decide if the source supports it, contradicts it, or says nothing.
Third, score: FActScore is simply the fraction of atomic facts that the source supports. A passage with 20 facts where 18 check out scores 0.9.
The big payoff is localization. You learn which claims were hallucinated, not just that the passage was 'somewhat wrong.' The catch: it only works when the knowledge source covers the topic, and a lookup per fact is slow and costly.
**FActScore** evaluates the factual accuracy of long-form generations through three stages.
First, decompose. The text is broken into atomic facts, the smallest independently verifiable claims. 'Marie Curie was born in Warsaw in 1867' becomes two facts. Decomposition is usually done by an LLM prompted to extract claims.
Second, verify. For each atomic fact, retrieve relevant passages from an external knowledge source such as Wikipedia. An NLI model or LLM then labels the fact as supported, contradicted, or not-enough-info. Grounding on an external source, not the model's own text, avoids self-referential grading.
Third, score. FActScore is the fraction of atomic facts that are supported:
$$\text{FActScore} = \frac{\#\,\text{supported atomic facts}}{\#\,\text{total atomic facts}}$$
Granularity is the key win. Per-claim scoring localizes exactly which claims hallucinated. Holistic paragraph scoring cannot do that.
The limitations are real. Verification quality collapses for topics the source covers poorly, like recent events or niche people. The pipeline is also expensive, since every fact needs retrieval plus a classification call. Decomposition itself can be noisy, splitting unevenly or dropping context.
Quick reference
TL;DR
FActScore decomposes long-form text into atomic facts, verifies each against an external knowledge source, and scores the fraction supported. Strength: per-claim localization. Weakness: cost and source coverage.
Imagine a friend tells you a long story about a celebrity. Instead of asking 'was that whole story true?', you write down every single small claim separately: born in this city, won this award, married this person. Then you check each tiny claim one at a time against a trusted encyclopedia. For each one you say yes it matches, no it contradicts, or the book does not say. Your final grade is just the fraction of small claims that matched. The clever part is that breaking the story into bite-size pieces tells you exactly which sentences were made up, not just that something somewhere was wrong. The catch: if the encyclopedia has nothing about that person, you cannot check the claims, so your grade becomes unreliable.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Evaluating claims about very recent events or fast-moving topics The knowledge source lags reality, so most facts land in not-enough-info and the score loses meaning.
Long-tail or proprietary domains with poor reference coverage Wikipedia or search returns little, so verification quality degrades and the score reflects coverage, not truth.
High-volume routine eval under a tight compute budget Per-fact retrieval plus a classification call across thousands of claims is far costlier than a single judge pass.
Short outputs with only one or two checkable claims Decomposition overhead buys little; a direct exact-match or simple grounded check is cheaper and just as informative.
Quick scan · 2–4 min
Everything important, quickly.
5 min: the three stages (decompose, verify, score), the atomic-fact example, the localization win over holistic scoring, then the three hard limits: decomposition reliability, knowledge-source coverage, and per-fact cost.
Real-world examples
Real products, models, and research that use this idea.
1RAGAS implements claim-level faithfulness by decomposing answers into statements and entailment-checking each against retrieved context, the same pattern FActScore introduced.
2TruLens groundedness scoring breaks responses into claims and verifies each against source context rather than scoring holistically.
3Modern factuality benchmarks evaluate frontier models like GPT-5.5 and Claude Opus 4.7 on biography generation using atomic-fact pipelines descended from FActScore.
4Long-form QA leaderboards report supported-fact fractions per response so teams can localize which specific claims a model fabricated.
5Production RAG eval stacks cache Wikipedia retrievals and sample atomic facts to keep per-fact verification cost manageable at scale.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you handle atomic facts that the knowledge source returns not-enough-info for?
Sign in or go Pro to see the approach for this follow-up.
QHow do decomposition errors bias the final FActScore, and how would you measure that?
Sign in or go Pro to see the approach for this follow-up.
QHow would you adapt FActScore for a domain with no Wikipedia coverage, like internal company docs?
Sign in or go Pro to see the approach for this follow-up.
QWhy might FActScore disagree with human judgment even when every fact is correctly labeled?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Describing FActScore as a holistic 'is this paragraph accurate' judgment. It is the opposite: per-claim decomposition and verification, and the score is the fraction of atomic facts supported.