What semantic property does BM25 capture that dense embeddings tend to miss?
BM25 rewards exact rare-term matches (IDs, proper nouns, code symbols) that dense embeddings smooth away into nearby concepts.
Imagine searching a library two ways. The first librarian looks for books that share the actual rare words you said, like the exact serial number on a part or the exact name of a person. The second librarian works from the vibe of your sentence. Ask for cheap weekend getaways and she finds books about budget trips even if the word cheap never appears. She is great at meaning, but if you ask for serial number A7-9921 she might bring you any technical manual that feels similar. Keyword search is the first librarian. Dense search is the second. Real systems hire both and let each do what it does best.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Embedding-based retrieval is so dominant in the modern RAG stack that it is tempting to treat BM25 as a legacy keyword scorer kept around for backward compatibility. That framing is wrong, and getting it wrong is one of the more common failure modes when teams ship their first production retrieval system.
The right framing is that BM25 and dense embeddings capture two different kinds of similarity, and neither one strictly dominates the other. BM25 measures lexical overlap weighted by rarity. Dense embeddings measure distributional similarity learned from large-scale contrastive training. Each retriever wins decisively on the queries the other is structurally bad at.
The interview question hides this in a simple shape: which property does BM25 capture that dense embeddings miss? The answer is exact term and rare-token matching, and the deeper point is that production retrieval systems are hybrid because the failure modes are complementary.
Why dense embeddings smooth rare tokens
Modern embedding models are transformer encoders trained with a contrastive objective. The training signal pulls semantically similar pairs together in vector space and pushes random pairs apart. That objective is exactly what gives dense retrieval its paraphrase superpower: a model trained on millions of related sentence pairs learns that cancel my subscription and how do I unsubscribe belong in the same neighborhood.
The same training pressure works against rare-token preservation. The contrastive loss does not care which exact tokens you used; it only cares that your sentence ends up near other sentences with the same meaning. A query containing the identifier INC-48291 looks, to the embedding model, like a generic short string with an alphanumeric tag. The model has no incentive to keep that exact substring as a load-bearing dimension. Instead it tends to cluster the whole query near other queries that share the same syntactic shape, like a short ticket lookup, regardless of which specific incident number was named.
This is not a model-quality problem you fix by going from a 768-dim encoder to a 3072-dim encoder. Bigger embeddings encode more semantic structure but they still optimize the same contrastive objective, so they still smooth rare tokens. The fix is structural: add a retriever that scores exact token overlap, and combine the two.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- GitHub code search and Sourcegraph rely on BM25-style sparse signals for exact symbol lookup, because dense-only retrieval blurs function names and identifiers.
- Elastic, Vespa, and OpenSearch ship hybrid retrieval out of the box, pairing BM25 with an ANN index over embeddings like OpenAI text-embedding-3-large or Voyage v3.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does Reciprocal Rank Fusion outperform a simple weighted sum of BM25 and cosine scores?
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Assuming dense embeddings strictly dominate BM25. They lose on rare-token queries where exact match is the signal that matters.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.