Click any words you think contain an error. Click again to unmark.
The first stage fetches only 3 candidates, then reranks those same 3 down to k=5. A reranker can only reorder what it is given — over-fetch a wide N (50-100) so it has room to surface the right chunk.
Imagine you hire a careful expert to pick the five best job applicants. But the front desk only hands the expert three resumes out of thousands, picked by a quick keyword filter. No matter how good the expert is, they can only choose among those three — the perfect candidate sitting at number twelve in the quick filter never reaches their desk. That is the bug here. The careful step, the reranker, is wasted because the rough first step handed it a tiny, already-cut pile. The fix is to let the quick filter pass forward a much wider pile, say fifty resumes, so the expert actually has room to find the best ones. A second reviewer only helps if the first reviewer gives them enough to work with.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Spot-the-error questions in RAG usually hide a conceptual misunderstanding behind a plausible-looking few lines of code, and this one is a clean example. The code reads like a correct two-stage retriever: embed, search, rerank, slice. Everything is there in the right order. The defect is a single number, top_k=3, that quietly defeats the entire architecture.
What makes this a good interview probe is the symptom attached to it: adding the reranker barely changed answer quality. A weak candidate sees working code and shrugs. A strong candidate recognizes that symptom as a fingerprint — it is precisely what a recall-starved funnel produces — and traces it back to the first-stage fan-out. The deep dive below explains why the recall ceiling set by N dominates everything downstream, why the reranker is powerless to fix a pool it did not get to see, and how to size the funnel so the expensive stage actually earns its cost.
What the funnel is supposed to do
A retrieve then rerank pipeline splits one hard problem into two stages with opposite priorities. The first stage is a cheap vector search over the whole index. Its job is recall: surface a candidate pool wide enough that the genuinely relevant chunk is somewhere inside it, even though a raw dot-product ranking is too crude to put it first. The retrieval score is just a similarity between the query and each candidate vector:
That score is fast to compute over millions of vectors but blind to the fine-grained query-document interactions that decide true relevance. So the ordering near the top is noisy — the right chunk might land at rank 8 or rank 12.
The second stage repairs that. A cross-encoder reads each query-candidate pair jointly and produces a far more accurate relevance score, reordering the pool so the genuinely relevant chunks rise to the top. It is expensive per candidate, which is why it runs only on the pool, never the corpus. The division of labor is recall then precision: the first stage decides what is eligible, the second decides the final order. Both halves are necessary, and the seam between them is the candidate count N.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Cohere Rerank's own guidance is to retrieve a wide candidate set first, often 50-100 documents, then rerank down to a handful for the prompt.
- LlamaIndex pipelines set a large similarity_top_k on the retriever and a small reranker top_n precisely to keep the funnel wide then narrow.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you choose the first-stage N rather than guessing 50 or 100?
Sweep N on a labeled query set and plot retrieval recall@N; set N at the plateau, since past that point you only add reranker cost for recall you already have.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Fetching only top_k=3 from the first stage, so the reranker reorders three chunks it was already handed and can never recover the chunk ranked twelfth.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.