Explain context recall in RAG evaluation and why it is more expensive than context precision
Explain what context recall measures in RAG evaluation, how it is computed, what ground truth it requires, and why that requirement makes it more operationally expensive than context precision.
Context recall is the fraction of reference-answer claims that the retrieved chunks support. It needs a gold answer per query, which makes it costlier than reference-free context precision.
Imagine an open-book exam. The retriever's job is to flip to the right pages before the model writes its answer. Context recall asks: of all the facts the perfect answer needs, how many were actually on the pages we opened? To grade that, you need the perfect answer written down in advance, then you check each fact against the open pages. That perfect answer is the expensive part: someone has to write or curate it for every single question. Context precision is cheaper because it only asks 'are the pages we opened on-topic?' You can eyeball that without knowing the perfect answer at all. So recall tells you what you missed, but only if you already paid someone to define what 'complete' means.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
A retrieval-augmented generation system has two stages that can each fail independently: the retriever fetches chunks from a knowledge base, and the generator writes an answer grounded in those chunks. Retrieval-side evaluation splits into two metrics that look symmetric but are operationally very different. Context recall asks whether retrieval surfaced everything the answer needed. Context precision asks whether what retrieval surfaced was actually relevant.
This question is hard because the interesting part is not the definitions, it is the cost asymmetry between them and what that asymmetry implies for how you build an eval program. Recall demands a gold reference answer for every query; precision needs nothing but the query and the chunks. That single difference cascades into how broadly you can measure each metric, why recall caps the whole pipeline, and how you bootstrap a labelled set without burning months of annotation effort.
The two metrics sit at the same point in the pipeline, the retrieval step, but they audit opposite failure modes. Precision catches noise: chunks you retrieved that you should not have. Recall catches gaps: chunks you needed that you never fetched. A retriever can score perfectly on one and terribly on the other, which is exactly why you want both numbers. The trap is assuming that because they are reported side by side in the same RAGAS report, they cost the same to produce. They do not, and a senior engineer who understands why designs a measurably cheaper eval program.
What context recall actually measures
Context recall is a coverage metric. It answers a precise question: of all the information a correct answer requires, what fraction did the retriever actually pull back? It is scoped entirely to the retrieval step, before the generator writes a single token.
The subtlety is in the phrase 'information a correct answer requires.' You cannot know what was required unless you already know what a correct answer looks like. That is why recall is anchored to a reference answer, the gold standard. The reference answer defines the target; recall measures how much of that target the retrieved context can support.
Contrast this with faithfulness, which people often conflate with recall. Faithfulness asks whether the generated answer is grounded in the retrieved context. Recall asks whether the retrieved context contains the gold claims in the first place. One is downstream of the generator, the other is upstream of it. Mixing them is the most common conceptual error on this topic.
There is a second, subtler distinction against answer relevance. Answer relevance scores whether the generated response addresses the question; it says nothing about whether the supporting evidence was retrieved. You can therefore have a relevant, faithful answer that is silently incomplete because recall was low and the generator simply never had the missing fact to include. Recall is the only retrieval-side metric that quantifies that silent incompleteness, which is what makes it the diagnostic of last resort when an answer looks fine but a user complains it left something out.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS computes context recall by claim-level attribution against a ground-truth answer, and ships synthetic test-set generation to bootstrap that gold set.
- TruLens reports context relevance (a precision-style, reference-free signal) separately from groundedness, reflecting the same cost asymmetry.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you bootstrap a labelled gold set for context recall without months of manual annotation?
Generate-then-verify: use a strong model to draft reference answers from source documents, then have humans verify and edit a stratified sample. Seed queries from real production logs so the distribution matches, and re-curate as the corpus drifts.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Confusing recall with precision, or claiming both are reference free. Context recall needs a gold answer per query to define what complete retrieval looks like, and that annotation cost is the whole point.
60 second bullets to scan on the way to the call.
Definition of context recall as coverage of reference-answer claims
The claim decomposition and attribution computation
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.