Pick an embedding dimension for a 50M-doc index with a 10 GB RAM budget

Start with the math. 50 million documents at a 1024-dim float32 vector each is `50e6 × 1024 × 4 bytes = 205 GB`. Your budget is 10 GB. The raw plan does not even come close. **Three levers** shrink the per-vector size. - **Truncation** with a Matryoshka-trained model: chop to 256 or 512 dims at small recall cost. - **Scalar quantization** to int8: 1 byte per dim instead of 4, with a small recall hit. - Binary quantization to 1 bit per dim: 1/8 byte per dim, with a 5-10% recall hit. You usually need to combine levers. A realistic landing: a 1024-dim Matryoshka model quantized to binary uses `50e6 × 1024 / 8 = 6.4 GB`, which leaves room for HNSW graph edges. Always validate the choice with a recall benchmark on a labeled query set before you ship.

**Storage math first.** 50M docs × d dims × bytes per dim must land below 10 GB minus HNSW graph overhead (roughly 2x the raw vectors). At float32 (4 bytes/dim) you can afford about d = 25, which is unusable. So quantization or truncation is mandatory. The lever menu in 2026. - **Int8 quant** (1 byte/dim): d ≤ ~100 within budget. Recall drop is small (1-3%). - **Binary quant** (1 bit/dim): d ≤ ~800 within budget. Recall drop is 5-10%. - Matryoshka truncation: cuts dim cleanly with small recall loss on Matryoshka-trained models. The realistic landing combines two levers. A 1024-d Matryoshka model with binary quant uses `50e6 × 1024 / 8 = 6.4 GB` raw. Which leaves ~3 GB for the HNSW graph at a 2x overhead, tight but workable. The alternative is DiskANN, which trades RAM for SSD latency. Final answer: 1024-d Matryoshka + binary quant + HNSW, validated against recall@10 on a labeled set with a documented quality budget.