Compute recall@5 and MRR for this labeled retrieval result
A query has exactly 4 relevant documents in the corpus. The retriever returns 5 results in ranked order; positions 2 and 4 are relevant, positions 1, 3, and 5 are not (positions are 1 indexed). Report recall@5 and MRR (reciprocal rank of the first relevant result) for this single query, each as a decimal rounded to 2 places.
Recall@5 = 2/4 = 0.50 counts how many of the 4 relevant docs landed in the top 5; MRR = 1/2 = 0.50 is the reciprocal rank of the first relevant hit. They coincide here by accident.
Imagine a treasure hunt with 4 hidden coins, and your detector beeps at 5 spots. Recall asks: of the 4 coins that exist, how many did you actually dig up? You found 2, so that's 2 out of 4. The other score, MRR, asks a totally different thing: how soon did you hit your first coin? Your first real coin was the second hole you dug, so the score is one-half. Both came out to one-half this time, but that's a coincidence — one counts how many coins, the other counts how soon the first one showed up.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
This looks like an arithmetic question, and the arithmetic is genuinely easy: both numbers are 0.50. But the reason it shows up in interviews is that the two metrics landing on the same value is a deliberate trap. A candidate who computes 0.50 and 0.50 and moves on has missed the point. A candidate who explains why they coincide, and constructs cases where they sharply diverge, has shown they understand what each metric actually measures.
Retrieval evaluation lives or dies on picking the right metric for the job, because a single number can flatter a retriever that is quietly failing the task. This deep dive computes both metrics carefully, dissects why they agree here, builds examples where they pull apart, and connects each to the retrieval scenario it is built for.
Computing recall@5 from first principles
Recall@k answers a coverage question: of all the documents that are relevant to this query, what fraction did the retriever place in the top k results?
The denominator is the total number of relevant documents in the corpus — here, 4. The numerator is how many of those relevant documents appear in the returned top-k list. The returned list has relevant documents at positions 2 and 4, which is 2 of them.
The most common error is to put the wrong number in the denominator. Recall does not divide by 5 (the number of results retrieved) — that would be precision@5, a different metric measuring how clean the returned list is. Recall divides by 4, the count of relevant documents that exist. The distinction matters: precision asks "how much of what I returned was good," recall asks "how much of the good stuff did I find."
Notice what recall@5 is blind to: position. The two relevant docs could have been at ranks 1 and 2, or at ranks 4 and 5, and recall@5 would read 0.50 either way. Recall counts presence in the window; it does not care where in the window. That blindness is the gap MRR fills.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS and BEIR benchmark suites report recall@k and MRR side by side because each catches failures the other hides.
- A known-item search like 'find the API doc for parseToken' is evaluated on MRR, since users want the single right page at the top.
What an interviewer would ask next. Try answering before peeking at the approach.
QIf your generator's prompt only fits the top-2 chunks, which of these two metrics should you optimize, and why?
Lean toward first-hit quality and how high the best chunks rank, so MRR (and precision@2) matter more than recall@k. With only 2 slots, a relevant doc that exists but ranks 5th never reaches the prompt, so recall over a wide k overstates what the generator can actually use. Optimize the rank of the best chunks within your real budget.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Computing MRR as 2 relevant out of 5 retrieved (0.40), confusing it with precision — MRR only cares about the rank of the first relevant hit, not how many were found.
60 second bullets to scan on the way to the call.
State the denominator recall@k divides by and why it is total relevant, not retrieved
Compute recall@5 for this example step by step
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.