MQA = Multi-Query Attention: every Q head keeps its own projection but all heads share one K and one V, shrinking the KV cache by n_heads at a measurable quality cost.
Picture a meeting room with twenty people asking different questions, each consulting their own personal filing cabinet. MQA tears out nineteen of the cabinets and tells everyone to share the one that remains. The questions still vary person to person, but they all consult the same source of information. That makes the room dramatically cheaper to run, because there is only one cabinet to maintain. The catch is that the single shared cabinet has less room to specialize, so subtle answers may drop in quality. MQA is that aggressive sharing trade, useful when you need to squeeze the last byte of memory out of a deployment.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
MQA is the most aggressive KV-cache reduction available in the conventional attention family. It is also a useful pedagogical anchor: GQA was invented to fix MQA's quality regression, and multi-head latent attention in DeepSeek V4 takes the same idea further. So understanding MQA is the gateway to the modern attention-variant landscape.
The acronym expands to Multi-Query Attention. The name is slightly misleading, because the queries are not what changed; they remain multi-head exactly as in MHA. What changed is the key and value side. Every query head still has its own projection, but all query heads share a single K head and a single V head per layer. There is only one K tensor and one V tensor in the KV cache regardless of how many query heads the model has.
This deep dive walks through the mechanics, the inference-cost argument, the quality picture, the historical adopters, why the field moved on to GQA, and where MQA still makes sense in 2026.
What stays plural and what collapses
Start with a concrete model. Llama-2 7B before the GQA era had 32 Q heads, 32 K heads, and 32 V heads per attention layer. The per-token KV cache for one layer is 2 * 32 * d_head * bytes. With d_head = 128 and FP16, that is 16 KB per token per layer, and across 32 layers, 512 KB per token.
MQA changes the K and V side to a single head each. So the same model under MQA would have 32 Q heads but only 1 K head and 1 V head per layer. The per-token KV cache becomes 2 * 1 * 128 * 2 = 512 bytes per layer, 16 KB per token across the whole model. A 32x reduction in KV bytes per token.
Q projections are unchanged. The model still produces 32 distinct query vectors per token per layer. The only difference is that all 32 queries look up into the same K, weight by the same scores, and aggregate from the same V. From a code perspective MQA is a one-line change to the attention kernel: gather K and V at index 0 regardless of which query head you are computing for.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Variant | K and V heads | KV cache size | Quality vs MHA |
|---|---|---|---|
| MHA | n_heads | Full | Baseline |
| MQA | 1 | 1 / n_heads | Measurable regression on harder tasks |
| GQA (G = 8) | G = 8 | G / n_heads | Within a fraction of a percent |
Real products, models, and research that use this idea.
- PaLM, the early Google decoder-only model, used MQA throughout to keep its KV cache small at scale.
- Falcon-40B from TII shipped MQA, which let it serve faster decode than equivalently sized MHA models.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does collapsing K and V hurt quality more than collapsing Q would?
Q determines which content each token searches for; K and V determine what is stored and retrieved. Sharing K and V across heads means every query head retrieves from the same memory. Sharing Q would be far worse because it would force every head to ask the same question.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Confusing MQA with GQA. MQA shares one K and V across the entire layer; GQA shares per group of heads.
60 second bullets to scan on the way to the call.
Expansion of the MQA acronym and what stays plural versus what collapses
Why decode bandwidth is the metric MQA optimizes
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.