Clarify the relationship between unfaithfulness and hallucination in RAG systems
A colleague says 'unfaithful and hallucinated mean the same thing: just different words for the same problem.' Explain why this conflation is incorrect and describe the practical consequences of treating them as equivalent in a RAG evaluation pipeline.
Faithfulness checks claims against the retrieved context; factuality checks them against the world. They diverge: an answer can be faithful but false or unfaithful but true, so you measure and debug each separately.
Imagine a student answering an open-book exam. Faithfulness asks: did every sentence come from the book in front of them? Factuality asks: is every sentence actually true? These are different questions. A student can copy the book perfectly, but if that page had a printing error, the answer is faithful yet wrong. A different student might ignore the book and write a correct fact from memory: that answer is true but unfaithful, because it is not backed by the source. So you cannot grade with one red pen. You need one mark for 'stuck to the book' and another for 'got the facts right'. Treating them as the same thing means you punish the right students and forgive the wrong ones.
Detailed answer & concept explanation~7 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
5 min: two reference points (context versus world), the four-cell matrix, the two dangerous off-diagonal cases, how to measure each axis, and which metric the use case demands.
Real products, models, and research that use this idea.
- RAGAS measures faithfulness via claim-level entailment against retrieved context, and answer correctness separately against a gold reference.
- TruLens splits the RAG triad into groundedness (faithfulness), context relevance, and answer relevance as distinct scores.
- FActScore decomposes a generation into atomic facts and verifies each against a trusted knowledge source, targeting factuality rather than grounding.
- Patronus and Galileo ship separate hallucination and groundedness detectors so teams can localize retriever versus generator failures.
- A 2026 legal RAG copilot built on Claude Opus 4.7 gates on faithfulness plus citation coverage, since an ungrounded but true claim is still non-auditable.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you build a metric that catches a faithful but false answer in production?
QWhy is claim-level decomposition better than a holistic faithfulness rating?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Collapsing faithfulness and factuality into one score. They have different reference points (context versus world), so one number cannot tell you whether retrieval or generation failed.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.