Define faithfulness in RAG evaluation and distinguish it from answer relevance

In a RAG system, the model first retrieves some documents, then writes an answer using them. **Faithfulness** is the metric that asks one simple question: is every claim in the answer actually supported by the retrieved documents? If the answer says something the documents do not back up, that is a faithfulness failure, even if the statement happens to be true in the real world. Faithfulness is about staying grounded in the context you were given, not about being correct in general. Answer relevance is a different metric. It asks whether the answer addresses the user's actual question. An answer can be perfectly grounded yet completely off-topic. The two are independent, so you measure both. An answer copied word for word from a document is faithful but may not answer the question (high faithfulness, low relevance). An on-topic answer that invents an extra detail is relevant but unfaithful (high relevance, low faithfulness). Faithfulness is the headline metric in RAGAS because hallucination, the model stating things the context never said, is the failure that most damages trust in RAG products.

**Faithfulness** measures whether every claim in a RAG answer is supported by the retrieved context. It checks grounding, not real-world truth. A true statement the context never mentioned still fails faithfulness. The standard way to compute it is claim-level entailment. You break the answer into atomic claims. You check each claim against the retrieved chunks. The score is the fraction of claims that are entailed. Faithfulness is: $$\text{faithfulness} = \frac{\text{claims supported by context}}{\text{total claims in answer}}$$ Answer relevance is a separate axis. It asks whether the answer addresses the user's question, regardless of grounding. The two are independent. You can be faithful but irrelevant. Copy a paragraph verbatim from a retrieved doc that never touches the question. Every claim is grounded, yet the user gets no answer. You can be relevant but unfaithful. Write a fluent on-topic answer that slips in one fact the context never stated. That single ungrounded claim is a hallucination, and it tanks faithfulness. That is why frameworks like RAGAS report them separately. Faithfulness catches hallucination. Relevance catches off-topic drift.