**Matryoshka** truncation lets you choose the embedding dimension at deploy time. The question is how to pick the dimension well.
The wrong way is to grab a number off a blog post (256, 512, 768) and use it without checking. The right d depends on three things specific to you: how good your embedding model is on your corpus, how strict your quality bar is. How much storage you can afford. None of those are universal, so a universal answer cannot be correct.
The right way is an empirical sweep. Build a labeled evaluation set drawn from your workload (a few hundred query-document pairs is enough). Truncate the stored vectors to each candidate dimension in a ladder like `{128, 256, 512, 1024, 1536, 3072}`.
Run retrieval at each dim and measure recall@10 (or whatever quality metric matches your product). Plot the curve of recall versus storage cost. Pick the smallest dimension whose recall clears your floor.
The curve has a recognizable shape: quality rises sharply at the smallest dims, then flattens. The right d is usually the elbow: the point where adding more dimensions stops paying for itself in quality.
**Sweep, plot, pick.** That is the entire procedure.
Build a labeled eval set drawn from your workload: a few hundred queries with known-good document IDs is enough for most retrieval workloads. Truncate the stored vectors to each candidate dimension in a ladder like `{128, 256, 512, 1024, 1536, 3072}`.
For each dim, run retrieval against the eval set and measure recall@10, or whatever metric matches your product surface (NDCG@10, MRR, precision@1, whatever your downstream actually cares about).
Plot recall@10 on the y-axis and storage cost per million docs on the x-axis. The curve has a predictable shape: steep gains at the smallest dims, then a flattening region where adding more dimensions costs storage without buying quality. The smallest d that clears your recall floor is the right answer.
The wrong answers fail in characteristic ways. 'Always use the max dim' wastes storage and ignores the point of Matryoshka. 'Use d=256 as the sweet spot' is a guess that happens to be right for some corpora and wrong for others. 'Pick d as a power of two' confuses index implementation details (no modern vector DB cares about power of two dims) with the actual cost-quality trade.
The principle: deploy-time dimensionality is an empirical choice, not a rule of thumb. Storage cost scales linearly with d, quality scales nonlinearly, and the right d is the one your own data tells you.
Quick reference
TL;DR
Sweep a ladder of truncation dimensions on a labeled eval set, plot recall@k versus storage cost, and pick the smallest dimension that clears your quality floor.
Imagine you have a giant photo and you need to shrink it to fit your phone wallpaper. You could pick a random small size, or you could try a few sizes side by side and see which one still looks good enough to recognize. Picking the right Matryoshka size for your data is the same. There is no universally correct number, because the right answer depends on your photo and your phone. The honest approach is to try several sizes against your own labeled examples, watch how well people still recognize the picture, and pick the smallest size that still passes your bar. Anyone who tells you 'always use 256' or 'always use the maximum' is guessing.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Your embedding model is not Matryoshka-trained Slicing a non-Matryoshka embedding destroys the geometry; the sweep is meaningless because the lower-dim slices are not valid embeddings.
You have no labeled eval data for your workload and cannot create any Without ground-truth labels, the sweep has nothing to optimize against and you are guessing again.
Storage cost is not the binding constraint and any quality drop is unacceptable If you are not trading anything for the dimension choice, the answer is the full dim and the sweep is wasted work.
Quick scan · 2–4 min
Everything important, quickly.
State the sweep plot pick procedure, justify each step, walk through why the distractors fail, and close on when to re-sweep.
Real-world examples
Real products, models, and research that use this idea.
1OpenAI's text-embedding-3-large documentation explicitly recommends evaluating the `dimensions` parameter against your own data rather than picking a default.
2Nomic Embed v1.5 ships truncation-quality curves in its release notes specifically so users can pick d empirically against their own workload.
3Snowflake Arctic Embed L 2.0 production case studies report different optimal d values (256-1024) across customers, reinforcing that the right answer is corpus-specific.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow large does the labeled eval set need to be for the sweep to give a stable signal?
Sign in or go Pro to see the approach for this follow-up.
QHow would you decide when to re-sweep after a corpus or model change?
Sign in or go Pro to see the approach for this follow-up.
QWhat does the storage versus quality curve typically look like, and what does the shape tell you?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Picking the truncation dim by reputation or round number. The right d depends on your corpus, your quality floor, and your storage budget. Only a sweep tells you the actual trade curve.