Flashcard: what is similarity search in RAG, and how does it differ from keyword search?

**Similarity search** is the retrieval step in RAG where you find the chunks most similar to the user's query. The trick is that the comparison happens in vector space, not in word space. Here is the picture. Every chunk in your corpus has been turned into a vector at indexing time. When the user asks something, you turn the question into a vector too, using the same embedding model. The retriever then asks: which stored vectors are closest to this query vector? The closeness is measured by cosine similarity, which is essentially the angle between two vectors. Why is this useful? Because closeness in this learned space tracks meaning. A query that says 'how do I cancel my plan' lands close to a chunk that says 'steps to terminate your subscription' because the embedding model learned that those sentences mean the same thing. A regular keyword search using BM25 or SQL LIKE would miss that chunk, because the literal words do not overlap. Similarity search wins on meaning. It loses on exact tokens, which is why hybrid retrieval combines both.

**Similarity search** is the algorithmic core of vector retrieval in RAG. Given a query vector and a corpus of chunk vectors, it returns the top-k chunks whose vectors are closest to the query under a distance metric. Cosine similarity is typical, with dot product or Euclidean distance as alternatives. The interesting property is semantic, not algorithmic. Because the embedding model has been trained on pairs of related texts, semantic similarity becomes geometric closeness in the embedding space. The phrases 'cancel my subscription' and 'terminate my plan' land near each other with zero word overlap. This is what makes RAG able to handle paraphrased questions and synonyms. It even handles cross-lingual queries when the embedding model supports them. The contrast with keyword search matters. BM25 and SQL LIKE compare token sets. They are excellent for exact strings (product names, error codes, identifiers) but blind to paraphrase. Similarity search inverts the tradeoff: strong on meaning, weak on exact tokens. The standard 2026 production pattern is **hybrid retrieval**, which fuses vector and BM25 scores with reciprocal rank fusion or a learned reweighting, then optionally reranks with a cross-encoder. This recovers both failure modes and is what every production quality RAG stack runs.