Name the metrics you would use to evaluate the retriever component on its own — separate from end to end answer quality — and explain why measuring retrieval in isolation matters for debugging a RAG system.
Score the retriever alone with recall@k, hit rate, MRR, and NDCG on labelled query-chunk pairs; isolating retrieval tells you whether to fix the retriever or the generator instead of tuning the wrong stage.
Imagine a cook who keeps serving bad dishes. Is it because the right ingredients never arrived from the pantry, or because the cook ruined good ingredients? You will waste days if you retrain the cook when the real problem is the pantry. Retrieval metrics check the pantry: did the right ingredients show up, and were they near the front of the shelf? If they did and the dish is still bad, the cook is at fault. Measuring the retriever on its own tells you which of the two to fix, so you stop tweaking recipes when the real bug is an empty shelf.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
2–4 min · Everything important, quickly.
Name the four retriever metrics and that they need a labelled query-chunk set. Then make the debugging case: a bad answer is either a retrieval miss (low recall@k or MRR, fix the retriever) or a generation failure (retrieval fine, fix the prompt). Show how isolating retrieval first prevents tuning the wrong stage, and how faithfulness layers on top to confirm a generation bug.
Real products, models, and research that use this idea.
- RAGAS pairs context recall and precision with faithfulness so teams can localize a bad answer to retrieval or generation.
- BEIR and MTEB rank embedding models by NDCG@10, which teams use to pick a retriever before building the generation stage.
- Production RAG teams gate embedding-model upgrades behind an offline recall@k and MRR check on labelled query-chunk pairs in CI.
What an interviewer would ask next. Try answering before peeking at the approach.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Tuning the generation prompt to fix bad answers without first checking retrieval. If the right chunk never reached the context, no prompt change can recover it.
60-second night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.