Your team must serve ANN search over 1B vectors at 1024 dimensions (float32) with a 50 ms p99 budget and moderate QPS. Walk through the capacity estimate for (a) fully in RAM HNSW and (b) DiskANN style serving from NVMe with a compressed in memory representation. Show the rough numbers and conclude which approach the economics favor.
In-RAM HNSW needs ~5 TB of RAM (multiple high-memory nodes). DiskANN needs ~50 GB RAM plus ~5 TB NVMe (one or two nodes). DiskANN wins by an order of magnitude at this scale.
Picture a library with a billion books. The first plan keeps every book on desks in the reading room so any book is grabbable instantly, but reading rooms cost a fortune per square foot, and a billion books need many huge rooms. The second plan keeps tiny index cards in the reading room and the actual books in a basement of cheap shelves; when you want a book, the index card tells you which shelf, and a runner fetches it in a few seconds. The basement plan is ten times cheaper, the books still get found, and the wait time is short enough that nobody complains. At a billion books, only one of these plans is affordable.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Capacity planning at billion-vector scale is where vector database design stops being abstract and starts costing real money. A team that gets the math wrong by 30 percent over-provisions and burns budget; a team that gets it wrong the other way OOMs in production. This question is also where the difference between in-RAM and disk-based ANN designs becomes economically forcing rather than a stylistic choice.
The rest of this section walks through the arithmetic for both designs, the price asymmetry that drives the decision, the latency check that confirms DiskANN can hit a 50 ms p99, and the boundary conditions where in-RAM HNSW remains the right call despite the cost. The number to anchor on is 4.1 TB of raw vector bytes; everything else follows from where you put them.
The 4.1 TB anchor and HNSW overhead
Raw vectors: 1 billion x 1024 dimensions x 4 bytes per float = 4,096,000,000,000 bytes, about 4.1 TB. That number is the floor for any design that keeps full-precision vectors anywhere.
HNSW adds the graph. At M=32 (a common production setting), each node stores its neighbor IDs across the layers. The exact overhead depends on layer counts and ID width, but a reasonable rule is 200 to 400 bytes per vector for adjacency at typical M values. For 1 billion vectors that is 0.2 to 0.4 TB.
On top of vectors plus graph, an engine adds metadata (per-vector IDs, payload pointers, deletion flags), allocator overhead, segment headers, and free space for ingest. Twenty to thirty percent overhead on the combined vectors-plus-graph total is a reasonable plan.
Putting it together: 4.1 TB (vectors) + 0.3 TB (graph) + 1 TB (overhead and headroom) = roughly 5-6 TB of RAM per replica. At two replicas for HA, that is 10-12 TB.
In cloud terms, 5 TB of RAM is 8 to 16 high-memory nodes (256 to 512 GB class) per replica. At list price for the high-memory instance families (r7iz, m6id, etc.), this is tens of thousands of dollars per month per replica. Managed vector database pricing reflects this: at billion scale on RAM-based engines, monthly bills are eye-watering.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Microsoft's DiskANN paper ships SIFT1B at >95 percent recall with single-digit ms latency on one machine
- Milvus and Vespa both ship DiskANN backends explicitly to serve billion-scale corpora from NVMe rather than RAM
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the recall and latency curve shift if you also enable PQ rescoring in DiskANN?
PQ codes give fast routing; rescoring against full vectors on disk improves recall but adds a few more SSD reads. The tradeoff is recall versus latency, tunable per query.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating raw vector bytes as the RAM budget and forgetting that HNSW adds graph adjacency and headroom, then under-provisioning by 25 to 50 percent.
60 second bullets to scan on the way to the call.
How to compute raw vector bytes from corpus size, dimension, and dtype
Why HNSW graph adjacency is not negligible at M=32
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.