Given a recall@10 budget and a storage budget, pick the right Matryoshka truncation level

**The right truncation dimension** under a storage budget and a recall floor is the smallest d that clears the recall floor on your own labeled eval set. That is it. The procedure is empirical and the answer is workload-specific. Storage scales linearly with d. A million 3072-dim float32 vectors take 12 GB; the same million at 768 dims take 3 GB. The 4x ratio is exact, not approximate. Search compute on a dot-product index also scales linearly with d, so a smaller d gives proportional latency improvements as well. The systematic procedure: build an eval set of a few hundred labeled queries from your workload. Truncate the stored vectors to each candidate dim in a ladder like `{128, 256, 512, 1024, 1536, 3072}`. Run retrieval at each dim and measure recall@10. Plot recall against storage cost. Pick the smallest d above your recall floor with a small safety margin. The wrong answers fail in instructive ways. 'Always 256' is a guess that happens to be right for some corpora and wrong for others. 'Half the original' is arbitrary. There is no theoretical reason to pick 1536 over 1024 or 768 absent measurement. 'Matryoshka is marketing' is just incorrect; OpenAI ships it under the `dimensions` API parameter and several open models do too.

**Truncate to the smallest d that meets your recall@10 floor on a labeled eval set.** Storage savings are linear in d. A 4x truncation gives exactly 4x storage savings, and search compute on a dot-product or HNSW kernel also scales roughly linearly with d. The procedure is a Pareto sweep over the cost-quality frontier. Build a labeled eval set drawn from your actual workload (a few hundred queries with at least one known-good doc each is enough for stable recall@10 measurement). Truncate the stored vectors to a geometric ladder of dimensions, typically `{128, 256, 512, 1024, 1536, 3072}` for a model with native dim 3072. Run retrieval at each level. Plot recall against storage. Pick the smallest d above your floor with a margin to absorb measurement variance. The distractors fail in instructive ways. 'Always 256' is a corpus-specific result repackaged as a universal rule. It works for short-query broad-domain workloads and fails for multilingual or technical corpora. 'Half the original' has no theoretical basis. There is no reason for the elbow to sit at exactly d/2. 'Matryoshka is marketing' ignores that OpenAI ships it on text-embedding-3-* via the `dimensions` parameter, and Nomic and Snowflake Arctic Embed do similarly. Several production deployments report 4-8x storage savings at 1-3 point recall cost. The principle: Matryoshka exists to make d a deploy-time decision per workload. Honoring the design means sweeping against the constraints you actually face.