Faithfulness checks that every claim in the answer is grounded in the retrieved context. Answer relevance checks that the answer addresses the question. They are independent.
Imagine an open-book exam. Faithfulness is the rule that you may only write things you can point to in the book in front of you. If you add a fact from memory that is not on those pages, you broke the rule, even if the fact is true. Answer relevance is a different rule: did you actually answer the question that was asked? You could copy a paragraph perfectly from the book (faithful) while never addressing the question (not relevant). Or you could write a great on-topic answer that quietly slips in a detail the book never mentioned (relevant but unfaithful). Good RAG systems need both: stay on the page, and stay on the question.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Faithfulness is the headline metric of RAG evaluation, and it answers a question that sounds obvious until you try to measure it: does the answer only say things the retrieved documents actually support? A retrieval-augmented generation system fetches context and then writes an answer from it. The whole promise of RAG is that the answer is grounded in real, retrieved evidence rather than the model's parametric memory. Faithfulness is how you put a number on that promise, and it is the first metric most teams reach for when a RAG product starts producing confident, wrong answers.
The common trap is to confuse faithfulness with correctness, and to confuse it with answer relevance. They sound similar in conversation, but they measure entirely different relationships, and treating them as one number throws away the diagnostic power that makes RAG evaluation useful in the first place.
This deep dive pins down exactly what faithfulness measures, why it is deliberately decoupled from real-world truth, how it differs from answer relevance, how a framework like RAGAS computes it in practice, and how the two metrics together let you localise a failure to the retriever or the generator. By the end you should be able to read a pair of scores and say where the bug lives.
What faithfulness actually measures
Faithfulness is the fraction of claims in the generated answer that are supported, or entailed, by the retrieved context. The unit is the claim, not the whole answer. You decompose the answer into atomic statements and check each one against the chunks the retriever returned. An answer that makes ten claims, nine of which are grounded and one of which is invented, scores 0.9, and that one ungrounded claim is exactly the thing you want to surface.
The subtle and important part is that faithfulness is decoupled from real-world truth. Suppose the retrieved context says nothing about a company's founding year, but the model writes 'founded in 1998', and that happens to be correct. Faithfulness still scores that claim as a failure, because the context did not support it. The model pulled it from its own parametric memory, which is exactly the behaviour RAG is supposed to suppress. The reverse also holds: if the retrieved document says the company was founded in 1997 and the model repeats that, the claim is fully faithful even though it is wrong in the world.
This decoupling is a feature, not a bug in the metric. It isolates one specific behaviour: did the generator stay inside its evidence? Mixing in world-truth would entangle two failures, a bad retriever serving wrong sources and a generator that strayed from its sources, and you would no longer be able to tell them apart. A separate factual-correctness check, against a gold answer, is what you use when you care whether the claim is true in the world. Faithfulness deliberately stays narrow so that it stays interpretable.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS reports faithfulness and answer relevancy as separate scores, computing faithfulness via claim-level entailment against retrieved context.
- TruLens ships a groundedness feedback function that is faithfulness under a different name, flagging answer sentences with no support in context.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does RAGAS actually compute a faithfulness score from a free-text answer?
Describe the two steps: decompose the answer into atomic claims with an LLM, then ask a judge to entailment-check each claim against the retrieved context. Score is the supported fraction. Mention judge noise on claim boundaries.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating faithfulness and relevance as the same thing. A faithful answer can be off-topic, and a relevant answer can still hallucinate claims the retrieved context never supports.
60 second bullets to scan on the way to the call.
Definition of faithfulness as claim-level grounding in retrieved context
Why faithfulness ignores real-world truth and only checks the context
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.