A colleague says 'unfaithful and hallucinated mean the same thing: just different words for the same problem.' Explain why this conflation is incorrect and describe the practical consequences of treating them as equivalent in a RAG evaluation pipeline.
Faithfulness checks claims against the retrieved context; factuality checks them against the world. They diverge: an answer can be faithful but false or unfaithful but true, so you measure and debug each separately.
Imagine a student answering an open-book exam. Faithfulness asks: did every sentence come from the book in front of them? Factuality asks: is every sentence actually true? These are different questions. A student can copy the book perfectly, but if that page had a printing error, the answer is faithful yet wrong. A different student might ignore the book and write a correct fact from memory: that answer is true but unfaithful, because it is not backed by the source. So you cannot grade with one red pen. You need one mark for 'stuck to the book' and another for 'got the facts right'. Treating them as the same thing means you punish the right students and forgive the wrong ones.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The colleague's claim sounds reasonable: both unfaithfulness and hallucination feel like 'the model said something wrong.' But they are wrong in different directions, against different yardsticks, and a serious RAG evaluator keeps them on separate axes. The conflation is one of the most common mistakes in production eval design, and it survives precisely because in the easy cases the two metrics agree. The agreement is a coincidence of the easy cases, not a property of the concepts.
Faithfulness measures whether a claim is entailed by the retrieved context that was fed to the generator. Factuality measures whether the claim is true in the world. Hallucination is simply a factuality failure: a claim that is false. Because faithfulness and factuality reference different ground truths, they can and do disagree. The whole point of this question is to see whether you can name the cases where they diverge and explain why that divergence matters for how you build and debug an eval pipeline.
The stakes are concrete. A team that believes the two are identical will choose one metric, optimize it, and ship a system that fails in exactly the way the chosen metric is blind to. The rest of this deep dive establishes the two reference points, enumerates the four-cell matrix, walks both off-diagonal failure modes, shows how to measure each axis with the right machinery, and explains how the use-case contract decides which metric is primary.
Two reference points, not one
The single most important idea is that faithfulness and factuality answer different questions about the same sentence.
Faithfulness asks: is this claim supported by the retrieved context? The yardstick is the documents the retriever handed to the generator. It is an entailment relation between the answer and the context, and nothing outside that context counts. A faithful answer never asserts anything the context did not.
Factuality asks a separate question: is this claim true in the real world? The yardstick here is reality, approximated by a gold answer, a curated knowledge base, or trusted external sources. Hallucination is the negative of factuality. A hallucinated claim is one that is false against that external yardstick, regardless of where it came from.
Because the yardsticks differ, you cannot infer one from the other. A claim grounded in context tells you nothing about whether the world agrees, and a true claim tells you nothing about whether the context supported it.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS measures faithfulness via claim-level entailment against retrieved context, and answer correctness separately against a gold reference.
- TruLens splits the RAG triad into groundedness (faithfulness), context relevance, and answer relevance as distinct scores.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you build a metric that catches a faithful but false answer in production?
Faithfulness against context cannot catch it by definition. Layer a factuality check against an external knowledge base or gold set, and monitor retrieval freshness so stale documents are flagged before they reach the generator.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Collapsing faithfulness and factuality into one score. They have different reference points (context versus world), so one number cannot tell you whether retrieval or generation failed.
60 second bullets to scan on the way to the call.
Faithfulness reference point versus factuality reference point
The four cell matrix and the two off diagonal cases
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.