You have 10M documents embedded as 3072 dim float32 vectors. You truncate to 768 dimensions using Matryoshka. Compute the savings in storage (express as a ratio: original storage / truncated storage).
Storage is linear in dimension, so 3072 → 768 gives 3072/768 = 4x savings. The recall cost for a well-trained Matryoshka model is typically under 5 recall@10 points.
Picture a row of identical lockers, each holding one filled-out form. If you shrink each locker to one quarter its old size, you fit four times as many lockers in the same room. That is the whole calculation. The arithmetic is just three thousand seventy two divided by seven hundred sixty eight, which equals four, because shelf space is a straight count of numbers written on each form. You cut the count of numbers per form by four. No hidden factors, no extra overhead. The interesting question is what you lose by shrinking the locker. For a form designed to be safely chopped short, the answer is surprisingly little, usually a small dip in search quality, not a collapse.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
This is a one-step calculation with a wider production story underneath. The storage savings ratio for a Matryoshka truncation is exactly the dimension ratio when the float dtype and vector count are held constant. For 3072-dim to 768-dim float32 vectors, the ratio is exactly 4x.
The interview value of the question lies less in the arithmetic and more in what a senior engineer says next. Storage savings translate to compute savings on simple index types and sub-linearly to HNSW query latency. Index overhead erodes the realized total savings on disk. Quality cost is workload-specific and needs measurement. Each of these caveats is small individually but they compose into the operational reality you would defend in a design review.
This answer walks through the calculation, the linear scaling that makes it exact, the secondary savings on compute and bandwidth, the index-overhead nuance, and the quality cost story.
The calculation and why it is exact
Storage cost for a corpus of N vectors at dimension d using a fixed float dtype is:
where b is the bytes per element (4 for float32, 2 for float16 or bfloat16, 1 for int8, 1/8 for binary). When N and b are held constant, the storage ratio between two dim choices is just the dim ratio:
The ratio is exact, not approximate, because there is no hidden per-vector overhead in the raw storage. Each vector is a flat array of 32-bit floats; the array length is the only thing changing.
In absolute terms for this scenario:
- Original: 10,000,000 × 3072 × 4 = 122,880,000,000 bytes ≈ 122.88 GB
- Truncated: 10,000,000 × 768 × 4 = 30,720,000,000 bytes ≈ 30.72 GB
- Difference: 92.16 GB recovered
The 92 GB is real money in any cloud provider's storage pricing. At typical 2026 cold-tier rates of around $0.01/GB-month, that is roughly $11/month per shard replica; on hot-tier RAM-backed pricing, it can be 10-50x more. The savings compound across replicas and backup copies.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI text-embedding-3-large at 3072 dim costs 4x as much to store as the same model truncated to 768 via the `dimensions` parameter; this is the headline benefit OpenAI advertises.
- Nomic Embed v1.5 release notes report 1-2 recall@10 points lost at 4x truncation on broad-domain benchmarks, validating the cost-quality trade.
What an interviewer would ask next. Try answering before peeking at the approach.
QIf you also quantize from float32 to int8, what's the combined storage savings ratio?
Truncation gives 4x; int8 quantization gives another 4x. The two stack multiplicatively, for a 16x raw storage reduction. Quality costs compose nearly additively for well-trained models: 1-3 points from truncation, under 1 point from int8 quantization.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Forgetting that storage savings are exact, not approximate, when both the float dtype and the vector count stay the same. The only variable is dimension.
60 second bullets to scan on the way to the call.
How does raw storage scale when you cut the dimension by a factor of four?
Under what assumptions is the storage-ratio argument exact rather than approximate?
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.