Match each embedding storage-reduction technique to its compression ratio and quality cost

Match pairs·Hard·4.0 · 0·~2 min

Attempt it

Matryoshka truncation (3072 → 768)

Scalar quantization (float32 → int8)

Binary quantization (float32 → 1 bit per dim)

Product Quantization (PQ, 8 subquantizers)

**Four techniques, four different trades.** The match exists because each technique compresses embeddings in a structurally different way. The consequences for storage, query mechanics, and quality are different in each case. **Matryoshka truncation** cuts the dimension count. Going from 3072 to 768 is a 4x storage saving because storage is linear in dim. Quality cost on well-trained Matryoshka models is typically 1-3 recall@10 points. Query mechanics are unchanged. The truncated vector lives in the same coordinate system as the full vector, and you still use cosine or dot-product distance. **Scalar quantization to int8** shrinks each number from 4 bytes (float32) to 1 byte (int8). That gives 4x savings without changing dimension. Quality cost is under 1 recall@10 point on most modern embedding models, because int8 preserves most of the float32 precision that matters for retrieval. Query mechanics are near-identical to float32. **Binary quantization** shrinks each number to a single bit: positive or negative sign. That gives 32x raw savings. Quality cost is larger, 5-10 recall@10 points on its own. The query metric changes too: distance becomes Hamming distance (count of differing bits), which is extremely fast on modern CPUs. Product Quantization (PQ) chunks the vector into subvectors, builds a learned codebook of representative subvectors for each chunk, and stores only the codebook IDs. Storage savings depend on subquantizer count and codebook size; 8 subquantizers with 256-entry codebooks gives roughly 8-32x raw savings. Quality cost is 3-8 recall@10 points. Query distance uses precomputed lookup tables, which is fast but trickier to implement.

**The four techniques cleave into two axes**: what they reduce (dim count vs bytes per dim) and how the query distance changes. **Matryoshka truncation (4x, ~1-3 point loss, linear).** Cuts dim count at deploy time via a slice. Storage and compute both scale linearly with dim, so 3072→768 is exactly 4x. Quality cost is typically 1-3 recall@10 points on well-trained Matryoshka models. Query mechanics are unchanged: same coordinate system, same distance metric. **Scalar quantization to int8 (4x, <1 point loss, near-free).** Compresses each float32 element to int8 via a learned per-dim min-max scale. Storage drops 4x without touching dim count. Quality cost is sub-1 recall@10 point on modern models because int8 preserves the precision that matters for retrieval. Query latency is comparable to float32 with SIMD kernels. **Binary quantization (32x, 5-10 point loss, Hamming distance).** Reduces each dim to a single bit: the sign. Storage drops 32x. Quality cost on its own is large enough that binary is usually paired with a reranker on top-k candidates using full-precision vectors. The query metric becomes Hamming distance (XOR plus popcount), orders of magnitude faster than float dot product on CPU. Product Quantization (PQ, 8-32x, 3-8 point loss, lookup tables). Splits the vector into M subvectors (typically 8 or 16), learns a codebook of K centroids per subvector (typically K=256), and stores each subvector as an 8-bit codebook ID. Query distance uses asymmetric distance computation with precomputed tables: fast, but more complex to implement. They stack. Matryoshka reduces dim; the three quantizations reduce bytes per dim. Matryoshka + int8 (16x), Matryoshka + binary + rerank (128x), and IVF-PQ + Matryoshka are common in billion-scale indexes.

Quick reference

TL;DR

Matryoshka 4x linear, scalar quant 4x near-free, binary quant 32x with 5-10 point recall drop and Hamming distance, PQ 8-32x with asymmetric distance tables and 3-8 point loss.

Memory aid

Easy to grasp

Picture four different ways to fit too many clothes into one suitcase. The first trick is taking out half your outfits and leaving them home. Same kind of clothes, just fewer of them. The second trick is rolling each shirt tightly instead of folding it flat. Same shirts, packed denser. The third trick is replacing every shirt with a tiny black and white sketch of it on an index card. Massive space savings, but you lost color and texture. The fourth trick is photographing each outfit and stashing only the photo along with a recipe card explaining how to recreate the look. Clever, complex, big savings. Each trick gives you a different amount of suitcase room for a different amount of lost detail.

Key concepts

Embeddings Vector Databases

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Match each technique by ratio and quality cost, dissect what each reduces (dim vs bytes per dim), name the query mechanics for each, and close on how they stack multiplicatively in production.

Side-by-side

Technique	Storage savings	Quality cost (recall@10)	Query mechanics
Matryoshka truncation (3072→768)	4x	1-3 points	Same as float; slice the vector
Scalar quantization (float32→int8)	4x	<1 point	Dequantize or int8 SIMD
Binary quantization (float32→1 bit)	32x	5-10 points	Hamming distance (XOR + popcount)
Product Quantization (8-16 subq)	8-32x	3-8 points	Asymmetric distance tables (lookup)

Real-world examples

Real products, models, and research that use this idea.

OpenAI text-embedding-3-large supports Matryoshka via the `dimensions` parameter; production deployments routinely pair it with int8 quantization downstream in Qdrant or Weaviate for 16x total savings.
Faiss IVF-PQ is the canonical billion-scale index implementation; Meta uses it across its embedding-based retrieval systems, with 16 subquantizers and 256-entry codebooks typical.
ScaNN (Google) and DiskANN (Microsoft) use PQ variants in production for web-scale retrieval, including disk-resident indexes that combine PQ codes with IVF clustering.
Qdrant ships binary quantization with a built-in rerank step using full-precision vectors; this is the canonical pattern for binary in production retrieval.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWalk through a billion-scale production stack that combines two or more of these.

QWhen would you choose binary quantization over PQ at the same savings ratio?

QHow does the quality story shift when you stack two techniques?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Confusing Matryoshka with scalar quantization. They both give 4x savings at small quality cost, but one reduces dim count and the other reduces bytes per dim. They stack.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.