You have an IVF index over 10M 768-dim vectors, with nlist = 4096. You measure recall@10 and average query latency while sweeping nprobe across {1, 4, 16, 32, 50}. Predict the qualitative shape of both curves.Recall climbs steeply then plateaus near ~0.95 by nprobe ~32; latency rises linearly. Pick the smallest nprobe that hits your recall SLO; everything past that is wasted CPU.
An IVF index is like a library where every book has been presorted into one of 4096 themed rooms. When you walk in with a query, you don't search every room; you only open the few rooms whose theme is closest to what you want. **nprobe** is how many rooms you open. Open 1 room: you might miss the book if it ended up in a neighboring room. Open 16 rooms: you almost always find what you want. Open 50 rooms: you waste a lot of time walking through rooms whose contents you didn't need. The catch is that walking each new room takes a fixed amount of time (the room is a fixed size), but the chance of finding new matches drops off fast after the first few rooms. So time goes up in a straight line and accuracy goes up in a curve that flattens out.
Detailed answer & concept explanation~3 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: IVF mechanics + concave recall curve + linear latency curve + production tuning rule + comparison points at nprobe in {1, 4, 16, 32, 50}.
| nprobe | Recall@10 (typical) | Latency multiple |
|---|---|---|
| 1 | 0.50-0.65 | 1x |
| 4 | ~0.80 | ~4x |
| 16 | 0.90-0.93 | ~16x |
| 32 | 0.94-0.96 | ~32x |
| 50 | 0.95-0.97 | ~50x |
Real products, models, and research that use this idea.
- Faiss IndexIVFFlat and IndexIVFPQ both expose nprobe as a runtime parameter; tuning sweeps are the canonical first step in any Faiss deployment.
- Milvus exposes nprobe per query for IVF-family indexes; production deployments at LinkedIn and Salesforce pin it after ground-truth sweeps.
- pgvector's ivfflat index exposes ivfflat.probes as a session parameter; raising it trades latency for recall in the standard concave versus linear way.
- Pinecone's older pod based tiers exposed nprobe-equivalent settings; the serverless tier abstracts them but the underlying curve is identical.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you pick nlist for a new corpus?
QWhat changes if you switch from IVFFlat to IVF-PQ?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Pushing nprobe higher and higher in pursuit of recall without noticing that the curve has plateaued; you pay linear latency for sub-percent recall gains.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.