Compute (show your math) the RAM cost of an HNSW index for 100M vectors at 1024 dimensions with M=32, float32 storage. Include both the vectors themselves and the graph edges. Note any practical assumptions a senior engineer would explicitly call out.
Per-vector: 4 * 1024 = 4096 bytes. Per-node edges: ~256 bytes at M=32 (2*M at base + hierarchy decay). Per-node total ~4352 bytes. 100M nodes ~= 435 GB. Pad ~20% for overhead, land ~500-525 GB.
Think of HNSW like a huge interconnected card catalogue that has to live entirely on a single desk. Each vector is 1024 numbers, each 4 bytes, so 4 KB per card. With 100 million cards that's 400 GB just for the cards themselves. The cross-reference web that lets the search hop quickly between cards adds another ~25 GB. Total: about 425-450 GB of raw cost, padded to ~500 GB for operating-system overhead. The catch: all of it has to stay on the desk (RAM) because the search hops hundreds of times per query, and walking to a filing cabinet (disk) for each hop would destroy performance. Once you cross this threshold on a single machine, you switch to DiskANN (lives on SSD), IVF-PQ (squeezes vectors 100x), or shard across machines.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Capacity planning for HNSW needs several layers at once: the per-node memory model, the production padding factors, the architectural implications of crossing single-instance RAM limits, and the alternative architectures available at that scale. Getting the headline number right is necessary; mapping it to the right architectural decision completes the plan.
The per-node memory model
Two storage costs per node.
Vector payload. At d=1024 float32, the vector is 4 * 1024 = 4096 bytes. This dominates the per-node cost at 1024+-dim. At lower dimensions (e.g., 384-dim BGE-small embeddings) the vector cost drops to 1536 bytes and the graph becomes proportionally more significant.
HNSW graph edges. The canonical Malkov-Yashunin implementation uses 2*M edges per node at the base layer and M edges at higher layers. Each edge is a 4-byte uint32 node ID (8 bytes if the implementation uses uint64 for multi-billion node support).
For M=32, 4-byte IDs:
- Base-layer edges: 2 * 32 * 4 = 256 bytes per node.
- Upper-layer edges: the level multiplier is 1/ln(M), so a node reaches each higher layer with probability about 1/M. The average per-node contribution from all upper layers is ~M * 4 / (M - 1) ~= 4 bytes.
Total graph overhead per node: ~260 bytes; 256-320 bytes once implementation headers are included.
Per-node total at M=32, d=1024: ~4352 bytes.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Architecture | Per-vector RAM | Per-vector SSD | Total for 100M @ 1024-dim | Per-query latency | Recall ceiling |
|---|---|---|---|---|---|
| HNSW (M=32, fp32) | ~4.35 KB | 0 | ~435 GB | <1 ms | ~0.97-0.99 |
| HNSW (M=32, SQ-int8) | ~1.28 KB | 0 | ~128 GB | <1 ms | ~0.96-0.98 |
| HNSW (M=32, BQ) | ~384 bytes | 0 | ~38 GB | <1 ms | ~0.92-0.97 (binary-aware) |
| IVF-PQ (pq_m=32) | ~32-64 bytes | 0 | ~5-10 GB | ~3-10 ms | ~0.85-0.92 |
| DiskANN | ~64-128 bytes (PQ) | ~4 KB | ~10 GB RAM + 400 GB SSD | ~5-15 ms | ~0.95-0.98 |
Real products, models, and research that use this idea.
- AWS r6i.32xlarge (1 TB RAM, ~$8/hr) is a typical instance choice for 100M-vector HNSW indexes.
- Pinecone serverless keeps data in object storage at every size and caches it in memory or on local SSD, hiding the cost cliff from users.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the per-node graph cost change if we switch to int8 SQ for the vectors?
Vector payload drops from 4096 to 1024 bytes (4x reduction). Graph cost stays the same (~256 bytes). Per-node total: ~1280 bytes. 100M nodes ~= 128 GB raw, ~155 GB after padding. Brings the workload comfortably onto a single 256-GB-RAM instance.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Forgetting that the full vectors must stay RAM-resident. HNSW graph traversal does distance computations at every hop, so any disk involvement would push per query latency from sub-millisecond to tens of milliseconds. This is what makes the ~400 GB number a hard constraint, not a soft one.
60 second bullets to scan on the way to the call.
Per-vector cost at d=1024 fp32: 4096 bytes
Per-node graph cost at M=32 with 2*M base + hierarchy: ~256-320 bytes
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.