Zenaique

Spot the errors in this capacity plan for a 200M-vector deployment.

Spot the error·Hard·4.0 · 0·~2 min·Asked atBrowserbaseFractal Analytics
Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.
TL;DR

Three errors: zero headroom over raw bytes (graph and overhead matter), sharding does not halve latency (logarithmic search plus fan-out tail), and PQ is lossy not lossless.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

Picture planning a road trip and budgeting only the gas to drive in a straight line, splitting the trip into two cars to arrive in half the time, and packing groceries by squashing them into smaller bags because squashing changes nothing about the food. None of those work. You need extra gas for traffic and detours, two cars do not magically halve the drive when they both have to wait at the same red lights, and squashing groceries actually damages them. Capacity plans fail in the same three ways: no headroom, false latency math, and pretending compression is free.

Key concepts

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Capacity-planning errors are some of the most consequential mistakes in vector database deployments because they manifest weeks after launch when traffic is real and budgets are committed. The three errors in this plan are textbook: a RAM budget without headroom, a sharding plan that confuses QPS with latency, and a compression strategy that treats lossy quantization as free.

Each has a clean mechanism, a quantitative explanation, and a known fix. The senior signal is being able to name all three from a brief plan, explain why each is wrong, and propose what the plan should have said instead. The rest of this section walks through each error in detail.

Error 1: raw bytes are a floor, not a budget

The plan computes 200M x 1536 x 4 bytes = 1.23 TB and stops there. That number is correct for raw vectors and irrelevant as a RAM budget.

HNSW graph overhead. At M=32, each node stores adjacency lists across log(N) layers. The bottom layer holds M neighbors; upper layers have a sparser fan-out. A reasonable rule of thumb is 200 to 400 bytes per vector for graph adjacency across all layers, accounting for ID width (8 bytes) and the average node count across layers. At 200M vectors, this is 40 to 80 GB. Not negligible.

Engine and allocator overhead. Vector storage in production engines is not a tight packed array. Allocator slop, segment headers, deletion bitmaps, ID indexes, payload pointers, and free space for ingest add 10 to 20 percent on top of the vector-plus-graph bytes. Compaction temporarily doubles the bytes of the segment being rewritten, so operational headroom must accommodate that.

Working budget. For 200M vectors at 1536 dims: 1.23 TB (vectors) + 0.06 TB (graph) + 0.2 TB (overhead and headroom) = roughly 1.5 to 1.6 TB per replica. The plan's number is off by 25 to 30 percent. That is the difference between a fleet that runs and one that OOMs on the first segment compaction.

Why the error keeps happening: raw byte math is easy and looks complete. Adding overhead requires knowing the index type and the engine's internals. Most plans that pass review without scrutiny on this point have skipped it. The fix is a default rule: never budget less than 1.25x raw vector bytes for HNSW workloads.

Error 2: sharding scales QPS, not p99
Error 3: PQ is lossy, not free
Why all three errors keep appearing
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.

Real products, models, and research that use this idea.

  • Pinecone publishes capacity tables that include graph and engine overhead, not just raw vector bytes, exactly to avoid the headroom mistake
  • Vespa documentation explicitly warns that increasing partition count amplifies tail latency, the same fan-out math as the sharding error
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QIf you actually want to halve p99, what should you do instead of doubling shards?
A

Tune ef_search down (accept a recall hit), upgrade hardware (faster CPU, more memory bandwidth), or move to a lower-dimensional embedding. Sharding for latency is the wrong tool; pick the tool whose mechanism actually shortens per-query work.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating raw vector bytes as a complete RAM budget, equating shard count with halved latency, and assuming product quantization preserves recall exactly.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • How HNSW graph adjacency scales with M and vector count

  • Why operational headroom matters in addition to vector and graph bytes

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
HNSW vs IVF, when…
Flashcard·Medium