Compute (show your math) the RAM cost of an HNSW index for 100M vectors at 1024 dimensions with M=32, float32 storage. Include both the vectors themselves and the graph edges. Note any practical assumptions a senior engineer would explicitly call out.
Per-vector: 4 * 1024 = 4096 bytes. Per-node edges: ~256 bytes at M=32 (2*M at base + hierarchy decay). Per-node total ~4352 bytes. 100M nodes ~= 435 GB. Pad ~20% for overhead, land ~500-525 GB.
Think of HNSW like a huge interconnected card catalogue that has to live entirely on a single desk. Each vector is 1024 numbers, each 4 bytes, so 4 KB per card. With 100 million cards that's 400 GB just for the cards themselves. The cross-reference web that lets the search hop quickly between cards adds another ~25 GB. Total: about 425-450 GB of raw cost, padded to ~500 GB for operating-system overhead. **The key catch:** all of it has to stay on the desk (RAM) because the search hops hundreds of times per query, and walking to a filing cabinet (disk) for each hop would destroy performance. Once you cross this threshold on a single machine, you switch to DiskANN (lives on SSD), IVF-PQ (squeezes vectors 100x), or shard across machines.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Capacity planning for HNSW is a senior-level exercise because it requires holding several layers simultaneously: the per-node memory model, the production padding factors, the architectural implications of crossing single-instance RAM limits, and the alternative architectures available at that scale. Getting the headline number right is necessary; mapping it to the right architectural decision is what separates competent from senior.
The per-node memory model
Two storage costs per node.
Vector payload. At d=1024 float32, the vector is 4 * 1024 = 4096 bytes. This dominates the per-node cost at 1024+-dim. At lower dimensions (e.g., 384-dim BGE-small embeddings) the vector cost drops to 1536 bytes and the graph becomes proportionally more significant.
HNSW graph edges. The canonical Malkov-Yashunin implementation uses 2*M edges per node at the base layer and M edges at higher layers. Each edge is a 4-byte uint32 node ID (8 bytes if the implementation uses uint64 for multi-billion node support).
For M=32, 4-byte IDs:
- Base-layer edges: 2 * 32 * 4 = 256 bytes per node.
- Upper-layer edges: the level-promotion probability decays geometrically (typically 1/ln(M)), so the average per-node contribution from upper layers is ~M * 4 / (M - 1) ~= 4 bytes per layer above base, summed across all layers. For typical 100M-node indexes this adds ~50-80 bytes per node averaged.
Total graph overhead per node: ~300-340 bytes; round to 256-320 for the conservative estimate.
Per-node total at M=32, d=1024: ~4352 bytes.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Architecture | Per-vector RAM | Per-vector SSD | Total for 100M @ 1024-dim | Per-query latency | Recall ceiling |
|---|---|---|---|---|---|
| HNSW (M=32, fp32) | ~4.35 KB | 0 | ~435 GB | <1 ms | ~0.97-0.99 |
| HNSW (M=32, SQ-int8) | ~1.35 KB | 0 | ~135 GB | <1 ms | ~0.96-0.98 |
| HNSW (M=32, BQ) | ~256 bytes | 0 | ~25 GB | <1 ms | ~0.92-0.97 (binary-aware) |
| IVF-PQ (pq_m=32) | ~32-64 bytes | 0 | ~5-10 GB | ~3-10 ms | ~0.85-0.92 |
| DiskANN | ~64-128 bytes (PQ) | ~4 KB | ~10 GB RAM + 400 GB SSD | ~5-15 ms | ~0.95-0.98 |
Real products, models, and research that use this idea.
- AWS r6i.x32large (1 TB RAM, ~$8/hr) is a typical instance choice for 100M-vector HNSW indexes.
- Pinecone serverless transparently transitions to disk-backed storage past similar thresholds, hiding the cost cliff from users.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the per-node graph cost change if we switch to int8 SQ for the vectors?
Vector payload drops from 4096 to 1024 bytes (4x reduction). Graph cost stays the same (~256 bytes). Per-node total: ~1280 bytes. 100M nodes ~= 128 GB raw, ~155 GB after padding. Brings the workload comfortably onto a single 256-GB-RAM instance.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Forgetting that the full vectors must stay RAM-resident. HNSW graph traversal does distance computations at every hop, so any disk involvement would push per query latency from sub-millisecond to tens of milliseconds. This is what makes the ~400 GB number a hard constraint, not a soft one.
60 second bullets to scan on the way to the call.
Per-vector cost at d=1024 fp32: 4096 bytes
Per-node graph cost at M=32 with 2*M base + hierarchy: ~256-320 bytes
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.