Build the graph on GPU with NVIDIA CAGRA, then serve it from CPU RAM. Build cost collapses from hours to minutes without changing query infrastructure.
Imagine baking 14 hours of bread every night in a small home oven, but the bakery only has 2 hours before customers arrive. You will not finish in time. The fix is not to use a smaller oven or burn the bread hotter. You wheel in a giant industrial oven that bakes the same loaves in 20 minutes, then carry the finished loaves to the regular shelf for customers to pick up. The big oven is the GPU and CAGRA. The shelf is your normal CPU index. Customers (queries) never see the new oven and never have to change how they shop.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
When a team first hits a maintenance window wall on index rebuild, the instinct is to reach for knobs they already know. Tune ef_search. Switch the distance metric. Delete the oldest data. None of those touch the actual bottleneck.
The bottleneck in an HNSW or similar graph index is graph construction. Each new node has to find its nearest neighbors among the existing graph, which means many search walks per insert. At 500 million vectors, that is hundreds of millions of small graph traversals, each with branchy, cache-hostile memory access patterns. A general-purpose CPU is not the right silicon for that workload.
This deep dive walks through why graph construction is the bottleneck, how GPU-native algorithms like CAGRA exploit parallelism, why the resulting graph can be served from a regular CPU fleet, and how to think about the cost tradeoff.
What graph construction is actually doing
Building an HNSW graph means iterating through every vector in the corpus and, for each one, running an ANN search against the already-built portion of the graph to find its M nearest neighbors. Those neighbors become the new node's edges, with bidirectional pruning to maintain the graph's small-world properties.
For a corpus of N vectors, that is N searches against a structure that grows from empty to N. The amortized cost per insert is roughly logarithmic in N for a well-built graph, but the constant is large: many distance computations, many pointer chases, many cache misses.
On a CPU, the per-insert work is essentially serial. You can parallelize across inserts to some degree, but you have to be careful about edge contention and graph quality. In practice, CPU HNSW build saturates around 16 to 32 cores and does not scale past that. At 500M vectors with dimensions, real-world build times of 8 to 14 hours on a 64-core CPU are normal.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- NVIDIA cuVS ships CAGRA as the build path used by Milvus, FAISS, and Weaviate for GPU-accelerated index construction in 2026
- Pinecone's serverless tier rebuilds indexes on GPU-backed build pools and serves queries from CPU-backed read pools, decoupling the two cost curves
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat if the graph format mismatch prevents direct CAGRA to HNSW conversion?
Discuss the cuVS interop path that emits a conforming HNSW edge list, plus fallback strategies (build CAGRA-native, expose a CAGRA serve path) if the converter is unavailable in your stack.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Tuning ef_search or nprobe to compensate for an incomplete build. Those are query-time knobs and have zero effect on how long graph construction takes.
60 second bullets to scan on the way to the call.
What graph construction actually does and why it is parallel-friendly
How CAGRA exports an HNSW-compatible graph for CPU serving
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.