Flashcard: what is similarity search in RAG, and how does it differ from keyword search?
Similarity search finds the top-k chunks whose vectors sit closest to the query vector, so retrieval works on meaning rather than literal word overlap.
Picture every sentence as a point on a giant invisible map, where points sit close together when the sentences mean the same thing, even if they share no words. A user asks 'how do I cancel my plan'. We drop a pin where their question lives on the map, then look at the nearest neighbors. A document that says 'steps to terminate your subscription' lives almost at the same spot, so it shows up. A keyword search would never find that document, because the words 'cancel' and 'plan' are nowhere in it. Similarity search wins on meaning. The catch is that exact things, like product names or IDs, can drift apart in this map; for those, keyword search still wins. That is why production systems often run both and combine the scores.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Similarity search is the operation people mean when they say 'vector retrieval'. The phrase sounds technical and self-explanatory and turns out to hide three different decisions: what metric you use, what algorithm you use, and how you combine the result with other retrieval signals. Candidates who give the slogan definition (find the nearest vectors) and stop tend to recommend pure vector retrieval for products that actually need hybrid search, and they tend to set fixed cosine thresholds that misfire across queries.
The useful framing is to distinguish what similarity search is good at from what it is bad at. It is excellent at paraphrase, synonyms, and cross-domain meaning, because the embedding model has compressed those equivalences into geometric closeness. It is weak at exact tokens like product names, error codes, and identifiers, because the model is trained to abstract, not to memorize literal strings. Every interesting RAG decision around retrieval is downstream of this asymmetry.
This deep dive defines similarity search precisely, explains why cosine is the canonical metric, walks through approximate nearest neighbor as the algorithm that makes it practical, and names hybrid retrieval and reranking as the production defaults that fix its known failure modes.
What similarity search actually computes
Similarity search takes a query vector and an indexed set of corpus vectors and returns the top-k corpus vectors closest to the query under a chosen distance metric. The choice of metric matters less than people think in 2026, because embedding models are uniformly trained and normalized so that cosine similarity is the right comparison.
Cosine similarity is the cosine of the angle between two vectors. For normalized vectors (unit length), it equals the dot product. For unnormalized vectors, it equals the dot product divided by the product of the norms. The two are interchangeable on normalized embeddings, which is why most production code uses dot product on pre-normalized vectors for speed.
Why cosine and not Euclidean? Euclidean distance includes vector magnitude, which carries length and frequency artifacts that have nothing to do with meaning. A long passage and its short paraphrase can have different norms while pointing the same direction. Cosine isolates direction. Modern embedding models train explicitly for this property, which is why cosine works reliably and Euclidean does not.
The returned object is a ranked list of corpus vector IDs with their similarity scores. The vector database resolves IDs to chunk text and payload before returning. Everything downstream in the RAG pipeline operates on the resolved chunks.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Pinecone and Qdrant exposing native hybrid search that combines dense vector similarity with sparse keyword scores in a single query.
- Cohere Rerank 3 as a cross-encoder reranker on top of similarity search candidates, used widely in 2026 production RAG stacks.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is cosine similarity preferred over Euclidean distance for embedding retrieval?
Cosine ignores vector magnitude; embedding norms vary with text length and frequency, so cosine isolates direction (meaning) from norm (length artifact).
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Believing vector retrieval is strictly better than keyword retrieval. Exact tokens like product names, SKUs, and code identifiers often retrieve better with BM25; hybrid search exists because both have failure modes.
60 second bullets to scan on the way to the call.
What metric similarity search uses and why cosine is canonical
Why semantic similarity beats keyword search on paraphrased queries
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.