Pick the right way to fuse BM25 and dense scores before context assembly

MCQ·Medium·4.0 · 0·~1 min·Asked atCresta Microsoft Servicenow

Attempt it

The right answer is Reciprocal Rank Fusion (option 1). The other three are wrong in different ways. Adding raw BM25 and cosine scores (option 0) is wrong because the scores are on incompatible scales. BM25 can produce raw scores in the range 0 to 30 or more, depending on how long the documents are and how rare the matched terms are. Cosine similarity is bounded but typically clusters between 0.4 and 0.9. Adding them just makes BM25 dominate, and the dominance changes per query as BM25 score distributions shift. Concatenating the top-k from each retriever (option 2) throws away the most useful signal, which documents both retrievers agreed on. A document ranked high in both BM25 and dense is almost certainly more relevant than one that ranked high in only one, and concatenation does not see that. Averaging normalized scores (option 3) is a defensible second-best, but it is brittle. The normalization itself is the hard part. Min-max normalization is sensitive to outliers. Z-score normalization assumes a roughly Gaussian distribution. Whichever normalization you pick can be wrong for a particular query. Reciprocal Rank Fusion works on ranks rather than scores. For each candidate, the RRF score is the sum across retrievers of `1 / (k + rank_in_that_retriever)`, with k typically 60. A document at rank 1 in both retrievers gets the highest score. A document at rank 1 in one and rank 20 in the other gets a moderate score. It is scale-free, robust, and parameter-free in practice.

Option 1, Reciprocal Rank Fusion, is correct. The other three each fail in a way that motivates RRF specifically. **Adding raw BM25 and cosine scores** (option 0) is the textbook scale mismatch error. BM25 is essentially `sum of idf-weighted, length-normalized term overlap` and produces values bounded above only by query length and corpus statistics. Cosine similarity is bounded in [-1, 1] but in practice clusters in narrow bands like 0.4 to 0.9. Adding the two means whichever retriever happens to have larger numeric scale dominates the ranking, and the scale shifts per query. **Concatenating the top-k from each retriever** (option 2) loses the agreement signal entirely. The most reliable property of hybrid retrieval, that documents ranked high in both retrievers are very likely relevant, is invisible to concatenation. It also doubles the candidate set without any deduplication or scoring rationale. **Averaging normalized scores** (option 3) is the most respectable wrong answer. It can work, but it requires choosing a normalization (min-max, z-score, sigmoid) and the choice changes the outcome. Min-max is sensitive to outliers, a single very-high BM25 score compresses everything else into the bottom range. Z-score assumes a roughly Gaussian distribution that does not hold for BM25 across queries. The fragility is why score-fusion methods have repeatedly lost head to head benchmarks against RRF. **RRF** sidesteps all of this. Score formula: `RRF_score(doc) = sum over retrievers of 1 / (k + rank_in_that_retriever)`, k typically 60. Working on ranks means scales do not matter. A document at rank 1 in both retrievers gets the maximum possible score for that pair. A document at rank 1 in one and absent from the top-N of the other gets a moderate score. RRF is the default in Elastic, Vespa, Pinecone, Weaviate, OpenSearch, and Qdrant.

TL;DR

Reciprocal Rank Fusion works on ranks, not raw scores, so it sidesteps the fact that BM25 and cosine live on incompatible scales, and it is parameter-free in practice.

Memory aid

Easy to grasp

Imagine two teachers grading your essay on different scales. One gives marks out of 100, the other gives stars from 0 to 5. Adding 87 plus 4 is nonsense, different units. Averaging after a normalization is fragile because one teacher might happen to give very spread-out scores this term. The robust trick: ask each teacher to rank all the essays from best to worst. Then combine the rankings. An essay that came in first for both teachers is clearly best; an essay that came in first for one and tenth for the other is middling. Rank-based fusion does not care about the units. RRF is the standard recipe for this kind of combination.

Key concepts

Context engineering Retrieval-augmented generation

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Hybrid retrieval needs a fusion step. Two retrievers each produced a ranked list of candidates; the pipeline has to combine them into a single ranking before assembly. Three of the four options listed are wrong in instructive ways. The fourth, Reciprocal Rank Fusion, is the production default for reasons worth understanding.

Why raw-score addition fails

BM25 and cosine similarity are computed by different formulas with different bounds.

BM25 for a query against a document is roughly the sum over query terms of idf(term) × tf(term, doc) × length_norm. The raw output depends on how rare the terms are in the corpus and how long the matched document is. A query against a long-document corpus with common terms produces scores around 1-3. A query with rare terms against short documents can produce scores around 20-30.

Cosine similarity between two unit-norm embeddings is bounded in [-1, 1] and in practice clusters in narrow bands like 0.4 to 0.9 for retrieval workloads.

The consequence

Adding raw BM25 to raw cosine means the ranking is dominated by whichever retriever happens to have a larger numeric scale this query. That domination shifts per query as the BM25 score distribution changes with the term rarity profile. The ranking is unstable. A retrieval system with query-dependent dominance is not a production system.

Why concatenation and score averaging are also wrong

What RRF does and why it works

Production patterns and when RRF is not enough

Situations where this technique stops working.

2–4 min · Everything important, quickly.

python

def rrf_fuse(rank_lists: list[list[str]], k: int = 60) -> list[tuple[str, float]]:
    """Reciprocal Rank Fusion across N ranked lists.
    Each list is doc_ids ordered best-first.
    Returns (doc_id, score) sorted by score desc.
    """
    scores: dict[str, float] = {}
    for ranks in rank_lists:
        for r, doc_id in enumerate(ranks, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + r)
    return sorted(scores.items(), key=lambda kv: kv[1], reverse=True)

# Example
dense_top = ["d3", "d1", "d7", "d2", "d5"]
bm25_top = ["d1", "d3", "d9", "d4", "d2"]
fused = rrf_fuse([dense_top, bm25_top])
# d1 and d3 dominate because they ranked high in both lists

Real products, models, and research that use this idea.

Elasticsearch 8.x and OpenSearch 2.x expose `rrf` as the documented fusion operator for hybrid queries.
Vespa's hybrid retrieval guide uses RRF with k=60 as the default sample configuration.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhen would you reach for a weighted RRF instead of vanilla RRF?

When one retriever is known to be substantially better on the workload; introduce per-retriever weights as w / (k + rank) and tune on a labelled eval.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Adding raw BM25 and cosine scores. BM25 can range 0 to 30+ depending on document length and term rarity; cosine is bounded around 0.4 to 0.9. The sum is meaningless and varies wildly per query.

60 second bullets to scan on the way to the call.

State the RRF score formula
Explain why BM25 and cosine scores are on incompatible scales

Primary sources. Browse if you want the original framing.