How does DeepSeek's Multi-Latent Attention (MLA) compress the KV cache below GQA?
Explain how MLA works: what it caches, how K and V are reconstructed at attention time, and why it can compress 5-10× more than GQA while matching or exceeding GQA quality. What is the runtime cost?
MLA caches one shared low-rank latent per token instead of full per-head K and V, reconstructs heads via absorbed up-projections, and shrinks the cache about 10x versus MHA.
Imagine each token's keys and values are a long shopping list you must keep on a sticky note for later. GQA saves space by letting several shoppers share one list. MLA does something smarter: it notices the lists are all variations on a few common themes, so it stores just a short recipe code that captures the gist. Later, when a shopper needs their full list, they expand that recipe code back into the detailed list on the spot. The sticky note holds a tiny code instead of a long list, so you keep far less paper around. Expanding the code costs a little extra work, but fetching tiny notes from memory is so much faster that you come out ahead overall.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Multi-head Latent Attention is DeepSeek's answer to the central pain of long-context serving: the KV cache. By 2026 the cache, not the weights, is the binding memory constraint on most decode workloads. Decode is bandwidth-bound, meaning each generated token forces the GPU to stream the entire cache out of HBM, so the size of that cache sets the throughput ceiling. Every architectural trick in this space is ultimately a bet on how to store less per token without losing the information attention needs.
Grouped-Query Attention and Multi-Query Attention took the obvious route. They reduce the number of distinct KV heads, so there is literally less K and V to cache. MQA collapses to a single shared KV head; GQA picks a middle ground, often eight. This is structural sharing, decided by the architect before training, and it trades quality for memory along a fixed curve that the model has no say in.
MLA takes a different and more ambitious route. It treats the problem as learned low-rank compression. Rather than store full per-head keys and values, it stores a single small latent vector per token and reconstructs the heads on demand, with the reconstruction fused into the existing attention matmuls. The latent basis is learned jointly with everything else, so the model itself decides what to keep. This deep dive walks through what is cached, how reconstruction works, why the compression is so large, why quality holds, the RoPE wrinkle, and the kernel-engineering caveat that decides whether MLA actually pays off in production.
What MLA caches versus what MHA and GQA cache
In multi-head attention the cache holds full keys and values for every head. Its per-token, per-layer footprint scales with the number of KV heads times the head dimension, doubled for K and V. For a model with 128 heads at dimension 128 that is over thirty thousand numbers cached per token, per layer. GQA shrinks the head count to a handful, typically eight, cutting the cache proportionally but leaving the per-head dimension untouched.
MLA stores neither full set. For each token it computes a single shared latent vector, in DeepSeek's models on the order of 512 dimensions, and that latent is the only thing written to the cache. The full per-head K and V never persist anywhere in HBM during decode.
The latent is produced by a learned down-projection from the token's hidden state. You can read it as a compressed code that captures the information all heads will later need. Note the contrast in kind, not just degree: GQA still caches genuine keys and values, merely fewer of them, while MLA caches an abstract code from which keys and values are regenerated. The cache row is therefore one short vector per token rather than a stack of per-head keys and values, which is the entire source of the memory saving. This is also why the two techniques compose: you could in principle apply both, though MLA's latent is already aggressive enough that DeepSeek uses it alone.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- DeepSeek V2 introduced MLA in 2024, cutting KV cache to roughly 1/10 of MHA while improving benchmark scores over its GQA baseline.
- DeepSeek V3 and the R1 reasoning model carry MLA forward, enabling long reasoning traces within a tractable HBM budget on H800 clusters.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does MLA need a decoupled RoPE key alongside the latent?
Rotary position multiplies K by a position-dependent rotation. That rotation does not commute cleanly through the absorbed up-projection, so the score would no longer be a clean function of the latent. MLA splits off a small per-head RoPE-carrying key computed separately, keeping the rest of K inside the absorbable latent path.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Saying MLA just shares K and V heads like GQA does. MLA caches a learned low-rank latent and reconstructs per-head K and V from it, a fundamentally different mechanism.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.