Order the stages of a hybrid sparse plus dense query with reranking inside the database layer.

Order steps·Easy·4.0 · 0·~1 min·Asked atAi21 Infosys Zed

Attempt it

1Return the final top k documents with their fused and reranked scores
2Run ANN search over the dense index to fetch the top dense candidates
3Rerank the fused pool with a cross-encoder or late interaction model
4Fuse the two candidate lists with reciprocal rank fusion into one ranked pool
5Run BM25 over the inverted index to fetch the top sparse candidates
6Parse the query: extract keyword terms for sparse scoring and embed the text into a query vector

Hybrid search combines a keyword engine (BM25) with a semantic engine (vector ANN) because each catches what the other misses. BM25 nails exact terms and rare words; vector search nails synonyms, paraphrases, and concept matches. The pipeline walks six stages, in this order: - **Parse the query.** Extract the keyword terms BM25 needs and embed the full text into a query vector the ANN search needs. - **Run BM25.** Score documents by term frequency and inverse document frequency, return the top N (typically 100). - **Run ANN.** Search the dense index for the top N nearest neighbors of the query vector. This can run in parallel with BM25. - **Fuse the lists.** Reciprocal rank fusion combines the two ranked lists into one by their position in each list, not by their raw scores. This is the standard fusion method because the two scores live on different scales. - **Rerank.** A cross-encoder model reads each fused candidate alongside the query and assigns a more accurate relevance score. This step is expensive, so it only runs on the fused top 50 or so. - **Return top k.** Sort by the rerank score and emit the final top k. The slow, accurate model runs last on the smallest list. That ordering is the pipeline's whole point.

**The pipeline is a funnel of decreasing breadth and increasing cost per item.** Parse cheap, retrieve broad with two arms in parallel, fuse cheaply, rerank expensively on the survivors only. **Stage by stage.** - **Parse.** Tokenize for BM25; embed for ANN. Both consume the same query text but produce different artifacts. - **Retrieve in parallel.** BM25 hits the inverted index; ANN hits the HNSW or IVF index. Both return roughly top 100 candidates each. - **Fuse with RRF.** Reciprocal rank fusion scores each document as the sum of `1 / (60 + rank)` across the two lists. RRF works because it uses only ranks, which are scale-free; raw BM25 scores (5-40) and raw cosine scores (in [-1, 1]) cannot be added directly. - **Rerank.** A cross-encoder (BAAI bge-reranker-v2, Cohere Rerank, Voyage rerank-2) reads `(query, document)` pairs and returns a relevance score. Latency is ~10 ms per pair; running it on 50 fused candidates is 500 ms, runnable inside a request budget. - **Return top k.** Sort by rerank score, emit final top k. **Why the order is load-bearing.** If you ran the cross-encoder before fusion, you would score 200 items (100 per arm) instead of 50, doubling latency. If you fused before either arm returned, you would have nothing to fuse. If you skipped fusion and concatenated raw scores, BM25's larger numeric range would silently dominate. The pipeline shape is the cheapest correct order, which is why every production hybrid stack converges on it.

TL;DR

Parse, retrieve sparse and dense in parallel, fuse the lists by rank, rerank the fused pool with a cross-encoder, return top k. The cross-encoder runs last because it is the expensive step.

Memory aid

Easy to grasp

Picture a recipe search. Step one, you read the user's words: they typed 'spicy chicken curry'. You extract the keywords for the spice-rack search engine and you also turn the whole sentence into an embedding for the meaning search engine. Step two and three run at the same time. The keyword arm pulls recipes with the words 'spicy', 'chicken', 'curry'. The meaning arm pulls recipes that match the idea, including ones written as 'fiery birds in coconut sauce' that share no keywords. Step four, you combine the two shortlists by how high each recipe ranked on either arm, not by raw score, because the two arms produce numbers on different scales. Step five, a slow but smart judge reads the title and first paragraph of each recipe and re-orders the top dozen. Step six, you serve the top three. The slow judge runs last on purpose, because letting it score everything would take all day.

Key concepts

Vector Databases

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Hybrid search with reranking is the default 2026 retrieval stack. The pipeline assembles BM25 and dense vector search as complementary recall engines, fuses their results to remove score-scale incompatibility, then runs an expensive cross-encoder rerank on the survivors. The order of operations is not arbitrary; it is dictated by funnel economics, where each stage trades breadth for cost per item.

This walkthrough explains each stage, why it sits where it does, what the alternatives look like, and the operational details that show up in production.

Mental model: a six-stage funnel. Parse cheap, retrieve broad in parallel, fuse cheap on the ranks, rerank expensive on the survivors, emit top k. The slow accurate step sees the fewest items.

Why two retrieval arms

BM25 and vector search catch different things. BM25 scores documents by term frequency and inverse document frequency: a document with the rare query term in it scores high, even if the rest of the document is unrelated. This is exactly what you want for product SKUs, error codes, person names, and any query where the exact token matters.

Vector search scores by semantic similarity in the embedding space. A query 'how do I reset my password' matches a document titled 'account recovery procedure' even though they share zero words. This is what you want for natural-language questions, paraphrases, and concept queries where the exact term is unknown.

Neither arm dominates. A pure-BM25 system fails on paraphrases; a pure-vector system fails on exact tokens because embedding models compress information and lose specificity. Hybrid systems consistently outperform either alone across benchmarks (MS MARCO, BEIR, in-domain retrieval suites), which is why every production stack in 2026 fuses the two.

The two arms also have complementary failure modes. BM25 fails silently on synonyms; vector search fails silently on the absence of a critical term. Fusing the lists means a document needs to look good to either arm to make the candidate pool, raising recall without sacrificing precision.

Why fusion before reranking

Pool sizes and latency budgets

Reranker choice and production pitfalls

Why the pipeline order is not negotiable, and the 2026 reranker market

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Weaviate exposes a single hybrid query API that runs BM25 and vector search in parallel and applies RRF by default, with an alpha parameter to weight the two arms.
Qdrant 1.7+ ships a query API where sparse and dense queries fuse with RRF, then a rerank stage calls an external reranker like Cohere Rerank.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you size the rerank pool when latency budget shrinks from 1s to 200ms?

Drop pool size proportionally. At 10ms per cross-encoder pair, 200ms gives you 20 items. Either shrink the pool to top 20 fused candidates, or swap the cross-encoder for a faster late-interaction model that processes more items in parallel.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Running the cross-encoder over the whole corpus, or fusing before both arms return, or skipping fusion and concatenating raw scores. All three break the pipeline economics.

60 second bullets to scan on the way to the call.

Why BM25 and ANN are complementary rather than competing
Why fusion comes before reranking, not after

Primary sources. Browse if you want the original framing.