Match each vector index type to its primary characteristic.
HNSW is a graph index, IVF partitions with k-means, Flat is exact brute force, and PQ is a compression layer that composes with IVF or HNSW.
Imagine four ways to organize a giant library so you can find similar books fast. **HNSW** is a friendship web: each book points to a few of its closest friends, and you walk friend to friend until you find what you want. **IVF** sorts every book into one of a few hundred labeled boxes; you only open the boxes most likely to contain matches. **Flat** is simply walking past every shelf in order. It is honest and exact but unbearably slow once the library grows. **PQ** is a trick that shrinks each book into a tiny summary card so the whole library fits in a smaller room; you lose a little detail but save enormous space. PQ is usually used with IVF or HNSW, not alone.
Detailed answer & concept explanation~4 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: HNSW graph + IVF partitioning + Flat brute force + PQ compression layer + how IVF-PQ composes them + two stage retrieval at billion scale.
| Method | Type | Memory | Recall | Sweet spot |
|---|---|---|---|---|
| HNSW | Graph index | High (2-4x raw) | 0.99+ achievable | 1M to 100M vectors |
| IVF | Partition index | Low | 0.85-0.95 typical | 100M+, often with PQ |
| Flat | Brute force | Just the vectors | 1.0 (exact) | Under 100k vectors |
| PQ | Compression layer | 10-50x savings | Lower (quantization error) | Composed with IVF or HNSW |
Real products, models, and research that use this idea.
- Pinecone's managed service defaults to HNSW for indexes under 50M vectors and switches to compressed variants at higher scale.
- Milvus and Faiss both ship IVF-PQ as the canonical billion scale recipe, used inside LinkedIn, Salesforce, and many in house search stacks.
- Qdrant exposes HNSW with optional binary and scalar quantization for cost sensitive RAG deployments in 2026.
- Weaviate offers Flat as a per collection option for small datasets where indexing overhead is not worth it.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide nlist for an IVF index?
QWhat does Optimized PQ (OPQ) buy you over plain PQ?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Treating PQ as a peer of HNSW and IVF. It is a compression scheme that composes with one of them, not a standalone index family.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.