Complete the claim: the two-axis decomposition of RAG eval that diagnoses which layer failed.
Faithfulness measures whether the answer is grounded in the retrieved chunks; context recall measures whether retrieval found the chunks at all. They isolate generation vs retrieval failures.
Picture an open book exam. Two things can go wrong. First, the student writes an answer that contradicts the book, or makes up facts the book never stated. That is a writing problem. Second, the student turned to the wrong pages, so the book in front of them never contained the answer at all. That is a finding problem. RAG evaluation splits quality into exactly these two questions. Faithfulness asks whether the written answer stays true to the pages that were open. Context recall asks whether the right pages were ever found. Keeping them separate matters: if you only measure the final answer, a wrong answer tells you nothing about which step failed. Splitting the score tells you whether to fix the retriever or the generator.
Detailed answer & concept explanation~6 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: define both axes, map each to a pipeline layer, walk the four-quadrant diagnosis, then add precision / answer relevance and the LLM-judge caveat.
| Faithfulness | Context recall | Diagnosis | Where to fix |
|---|---|---|---|
| High | High | Working as intended | Nothing: this is the target |
| High | Low | Honest but ignorant (retrieval failure) | Chunking, embeddings, hybrid search, k |
| Low | High | Ignored good context (generation failure) | Prompt, context ordering, model |
| Low | Low | Broken on both ends | Retriever and generator both |
Real products, models, and research that use this idea.
- RAGAS computes faithfulness by extracting claims from the answer and verifying each against retrieved context, alongside context recall and context precision.
- TruLens scores the RAG Triad (context relevance, groundedness (faithfulness), and answer relevance) to localize failures by pipeline stage.
- Arize Phoenix traces per-query retrieval and generation scores so teams can spot whether low-recall or low faithfulness drives a regression.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow is faithfulness actually computed when there's no labeled ground truth for every query?
QContext recall needs a reference answer. How do you measure it in production where you have no labels?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Collapsing both axes into one end to end accuracy score. A single number hides whether the retriever missed the chunk or the generator ignored it, so you can't tell which layer to fix.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.