Doubling the embedding dimension from 768 to 1536 — what changes in storage, query latency, and PQ trainability?

Attempt it

Quick reference

TL;DR

Doubling d doubles storage and per-vector distance work (both linear in d) and makes PQ codebooks cover larger sub-vectors; fix by raising pq_m.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Think of each item in your database like a postal address written on a strip of paper, where the length of the strip is the dimension. Imagine doubling how long every strip is. Now every strip uses twice the storage in the filing cabinet, and reading a strip to compare it to another takes twice as long. For chunked-codebook shrinking (called PQ, which stores each strip's chunks as numbers from a tiny catalogue), keeping the same number of chunks while doubling the strip length means each chunk now covers twice as many characters, and a longer chunk is harder to summarize well with a small catalogue. The standard fix is to slice the longer strip into more chunks, rather than into the same number of fatter chunks.

Key concepts

Product quantization

Quick scan · 2–4 min

Everything important, quickly.

5 min: linear scaling of storage and compute with d, SIMD as a constant factor, memory bandwidth as a separate bottleneck, PQ codebook trainability, and Matryoshka truncation as the production escape hatch.

Side-by-side

Axis	Scaling with d	Mitigation
Raw storage (float32)	Linear (4 × d × N)	Use float16, int8, PQ, or BQ
Distance compute per pair	Linear (O(d))	SIMD reduces constant; slope is fixed
Memory bandwidth per pair	Linear (d × bytes loaded)	Compress vectors to reduce bytes/pair
PQ codebook quality at fixed pq_m	Degrades	Scale pq_m proportionally to d
Curse-of-dimensionality (recall)	Asymptotic, mild at d < 4096	Use natural-language manifold-aware embeddings

Real-world examples

Real products, models, and research that use this idea.

text-embedding-3-large at 3072 dims doubles storage versus text-embedding-3-small at 1536 dims; documented as a top cost surprise on Pinecone Serverless.
Cohere embed-v3 at 1024 dims sits at the lower-cost end; embed-v3-large at 4096 dims is the high-recall option for big budgets.
BGE-M3 supports multiple output dimensions (768, 1024) so teams can pick the cost-quality point at the embedding model layer.
Faiss PQ implementations default to pq_m chosen so that each sub-vector is 4-8 dims for d=128, scaling proportionally for larger d.
Matryoshka representation learning lets you truncate text-embedding-3-large from 3072 to 512 dims at the model level, cutting storage 6x while keeping most of the recall.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does memory bandwidth interact with the dimension scaling on HNSW versus IVF?

Sign in or go Pro to see the approach for this follow-up.

QIf you double d but the recall@10 only improves by 0.5 points, is the migration worth it?

Sign in or go Pro to see the approach for this follow-up.

QWhy does Matryoshka representation learning let you decouple model size from index size?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.