Complete the claim: the two-axis decomposition of RAG eval that diagnoses which layer failed.

Attempt it

Standard RAG evaluation decomposes pipeline quality into two orthogonal axes. The answer side metric (measuring whether the LLM's generated answer is grounded in, and not contradicted by, the retrieved chunks) is called . The context-side metric (measuring whether the retrieved chunks actually contain the information needed to answer the query) is called . When both axes are measured, a high faith / low-recall result indicates the system was honest but ignorant (a failure), and a low faith / high-recall result indicates the LLM ignored or contradicted available context (a failure).

Quick reference

TL;DR

Faithfulness measures whether the answer is grounded in the retrieved chunks; context recall measures whether retrieval found the chunks at all. They isolate generation vs retrieval failures.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Picture an open book exam. Two things can go wrong. First, the student writes an answer that contradicts the book, or makes up facts the book never stated. That is a writing problem. Second, the student turned to the wrong pages, so the book in front of them never contained the answer at all. That is a finding problem. RAG evaluation splits quality into exactly these two questions. Faithfulness asks whether the written answer stays true to the pages that were open. Context recall asks whether the right pages were ever found. Keeping them separate matters: if you only measure the final answer, a wrong answer tells you nothing about which step failed. Splitting the score tells you whether to fix the retriever or the generator.

Key concepts

LLM evaluation RAG system design Vector databases

Quick scan · 2–4 min

Everything important, quickly.

4 min: define both axes, map each to a pipeline layer, walk the four-quadrant diagnosis, then add precision / answer relevance and the LLM-judge caveat.

Side-by-side

Faithfulness	Context recall	Diagnosis	Where to fix
High	High	Working as intended	Nothing: this is the target
High	Low	Honest but ignorant (retrieval failure)	Chunking, embeddings, hybrid search, k
Low	High	Ignored good context (generation failure)	Prompt, context ordering, model
Low	Low	Broken on both ends	Retriever and generator both

Real-world examples

Real products, models, and research that use this idea.

RAGAS computes faithfulness by extracting claims from the answer and verifying each against retrieved context, alongside context recall and context precision.
TruLens scores the RAG Triad (context relevance, groundedness (faithfulness), and answer relevance) to localize failures by pipeline stage.
Arize Phoenix traces per-query retrieval and generation scores so teams can spot whether low-recall or low faithfulness drives a regression.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow is faithfulness actually computed when there's no labeled ground truth for every query?

Sign in or go Pro to see the approach for this follow-up.

QContext recall needs a reference answer. How do you measure it in production where you have no labels?

Sign in or go Pro to see the approach for this follow-up.

QIf faithfulness is high but users report wrong answers, what's likely happening?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.