Flashcard: what is a vector database, and why does RAG need one?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

A vector database is a data store optimized for fast approximate nearest-neighbor (ANN) search over high dimensional vectors. Examples include Pinecone, Weaviate, Qdrant, Milvus, pgvector on Postgres, and Chroma. RAG needs one because at query time you must compare the query embedding against millions or billions of chunk embeddings; a brute force O(N) cosine scan does not scale. Vector DBs use index structures like HNSW or IVF to return the top-k closest vectors in sub-linear time, typically tens of milliseconds even at scale. They also store the original chunk text and metadata alongside each vector, so retrieval returns the actual content, not just an ID.

Your answer

Quick reference

TL;DR

A vector database is a data store built around fast approximate nearest neighbor search over high dimensional embeddings, so RAG can fetch the top-k similar chunks in milliseconds.

Memory aid

Easy to grasp

Imagine you have a library where every book is placed at a coordinate based on its meaning, so books about similar topics sit close together on a vast multi-dimensional shelf. Now someone walks in and asks a question. You convert their question into a coordinate too, and you need to find the books closest to that point. Walking past every shelf would take forever in a real library. So the library has a clever map that lets you jump straight to the right neighborhood without checking every aisle. A vector database is that library plus that map. It stores millions of vectors and ships with a search algorithm that finds the closest ones in milliseconds. Without it, RAG would crawl, because comparing one query against a million embeddings the slow way is a slow way.

Key concepts

Vector databases Embeddings Retrieval-Augmented Generation

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

3 min: define the vector database as ANN index plus payload store, walk through why brute force scanning fails at scale, name HNSW and IVF, and list the 2026 vendor landscape.

Real-world examples

Real products, models, and research that use this idea.

Pinecone serverless hosting tens of billions of vectors with metadata filtering and sub-100ms search latency for production RAG.
Qdrant running in single binary mode with HNSW and on-disk vectors, common in self-hosted enterprise RAG deployments.
pgvector inside Postgres for teams that want vector search without operating a separate datastore, with HNSW added in pgvector 0.5+.
Weaviate combining vector search with native BM25 keyword search for hybrid retrieval in customer support RAG products.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does HNSW achieve sub-linear search, and what parameters control the recall latency tradeoff?

QWhen would you pick pgvector over a dedicated vector database like Pinecone or Qdrant?

QHow do you handle metadata filtered search without destroying ANN recall?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Assuming a vector database is just a regular database with a vector column. The defining feature is the ANN index (HNSW, IVF, ScaNN) that makes top-k search sub-linear; without it you have storage, not retrieval.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.