How does DeepSeek's Multi-Latent Attention (MLA) compress the KV cache below GQA?

Attempt it

Explain how MLA works: what it caches, how K and V are reconstructed at attention time, and why it can compress 5-10× more than GQA while matching or exceeding GQA quality. What is the runtime cost?

Free · 2 AI evals / day

Quick reference

TL;DR

MLA caches one shared low-rank latent per token instead of full per-head K and V, reconstructs heads via absorbed up-projections, and shrinks the cache about 10x versus MHA.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine each token's keys and values are a long shopping list you must keep on a sticky note for later. GQA saves space by letting several shoppers share one list. MLA does something smarter: it notices the lists are all variations on a few common themes, so it stores just a short recipe code that captures the gist. Later, when a shopper needs their full list, they expand that recipe code back into the detailed list on the spot. The sticky note holds a tiny code instead of a long list, so you keep far less paper around. Expanding the code costs a little extra work, but fetching tiny notes from memory is so much faster that you come out ahead overall.

Key concepts

Multi-head Latent Attention KV cache Low-rank compression Grouped-Query Attention

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

4 min: what MLA caches (latent) + reconstruction via absorbed up-projections + compression ratios vs MHA and GQA + why quality holds + decoupled RoPE + the kernel-fusion caveat.

Real-world examples

Real products, models, and research that use this idea.

DeepSeek V2 introduced MLA in 2024, cutting KV cache to roughly 1/10 of MHA while improving benchmark scores over its GQA baseline.
DeepSeek V3 and the R1 reasoning model carry MLA forward, enabling long reasoning traces within a tractable HBM budget on H800 clusters.
vLLM and SGLang added dedicated MLA attention kernels in 2025 so DeepSeek serving could realize the bandwidth win rather than stalling on the extra matmul.
Inference providers serving DeepSeek V3 cite MLA as the reason per-token decode cost stays low at 128k context versus comparable GQA models.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does MLA need a decoupled RoPE key alongside the latent?

Sign in or go Pro to see the approach for this follow-up.

QHow does the absorption of up-projections into W_Q and W_O actually save bandwidth?

Sign in or go Pro to see the approach for this follow-up.

QWhy can a learned latent match GQA quality despite caching far less?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Saying MLA just shares K and V heads like GQA does. MLA caches a learned low-rank latent and reconstructs per-head K and V from it, a fundamentally different mechanism.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.