Why does MQA underperform GQA when both share KV across heads?

Short answer·Medium·4.0 · 0·~3 min·Relevant atMeta Mistral AI

Attempt it

Multi-query attention (MQA) and grouped-query attention (GQA) both reduce KV cache by sharing K and V across query heads. Yet MQA usually degrades model quality noticeably while GQA does not. Explain the structural reason for that gap, and what knob makes GQA the safer compression point.

Free · 2 AI evals / day

Both MQA and GQA reduce the KV cache by sharing K and V across query heads. The difference is how much they share, and that difference cascades into a quality gap. MQA collapses everything: every query head in a layer reads the same K and V. The cache shrinks by a factor of num_heads (the largest possible reduction), but every head now indexes into the same memory. The parallel specialization that multi-head attention is designed to provide is mostly lost on the lookup side. GQA splits the query heads into G groups and gives each group its own K and V. The cache shrinks by a factor of `num_heads / G` (smaller reduction than MQA but still huge), and each group retains its own K, V subspace. So heads within a group share, but groups remain distinct. The knob is **group size G**. G=1 is MQA (maximum compression, biggest quality hit). G=num_heads is MHA (no compression, no quality hit). G=8 is the production sweet spot in most 2026 frontier models, capturing most of the cache savings with quality within noise of full MHA. Production examples: Llama-3 70B and Llama 4 Maverick both use 8 KV heads per 64 query heads.

MQA underperforms GQA because the granularity of K, V sharing determines how much head-level specialization survives. Multi-head attention's central insight is that different heads can learn to attend to different relations: positional adjacency, syntactic dependencies, coreference, document structure. Each head's specialization is most directly encoded in its (K, V) subspace, the 'shelf' it indexes into. When you collapse to a single shared K, V (MQA), every head reads from the same shelf. Queries can still differ, but every question lands in the same memory. GQA preserves a partial version of that diversity. With G groups, you have G distinct K, V subspaces per layer. Heads within a group share, but the groups remain genuinely different views of the input. Empirically this is enough to recover most of MHA's quality. The knob is group size G: - G=1: MQA. Maximum compression (factor of num_heads), biggest quality hit. - G=8: production sweet spot. ~8x cache savings (relative to MHA) and quality within noise of MHA. - G=num_heads: MHA. No compression, no quality cost. The quality vs G curve is convex: most of the recovery happens between G=1 and G=8, and additional groups beyond that give diminishing returns. That is why production frontier models in 2026 (Llama 4 Maverick, Claude Opus 4.7, Gemini 3.1 Pro, Mistral Large 3) converge on G in the 4 to 8 range.

Quick reference

TL;DR

MQA collapses all heads onto one K, V, killing lookup-side diversity. GQA preserves group-level K, V diversity via a group-size knob, keeping most quality at almost MQA's cache size.

Memory aid

Easy to grasp

Imagine a research team of 64 analysts. Multi-head attention gives each analyst their own personal library. MQA fires the librarians and forces all 64 analysts to share one library, much cheaper, but every analyst now looks at the same shelf, so their analyses become similar. GQA splits the analysts into 8 groups of 8 and gives each group its own library, almost as cheap as one shared library but with 8 different views into the source material. The group size is the knob.

Key concepts

Multi-query attention Grouped-query attention KV cache

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5m: why MHA's head diversity matters, how MQA's single shared K, V collapses lookup-side diversity, what GQA's group size G preserves, the convex quality vs G curve, and the production convergence on G in the 4 to 8 range.

Side-by-side

Property	MQA	GQA (G=8)	MHA
KV heads per layer	1	8	num_heads
Cache reduction vs MHA	num_heads x	num_heads / 8 x	1x (baseline)
Lookup-side diversity	None	8 subspaces	num_heads subspaces
Quality vs MHA	Measurable drop	Within noise	Baseline
Production usage in 2026	Rare (legacy)	Default	Encoders, small models

Real-world examples

Real products, models, and research that use this idea.

Llama-3 70B uses GQA with 8 KV heads per 64 query heads.
Llama 4 Maverick continues the G=8 pattern as the production default.
Claude Opus 4.7 and Gemini 3.1 Pro both use GQA at similar group counts.
Mistral Large 3 uses GQA combined with a sliding-window and full-attention layer hybrid.
The Ainslie et al. 2023 GQA paper empirically swept group sizes from 1 (MQA) to num_heads (MHA) and showed the convex quality curve.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy is the quality vs G curve convex rather than linear?

QHow would you choose G for a deployment with a fixed memory budget?

QHow does MLA in DeepSeek V4 sidestep the MQA vs GQA tradeoff?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating MQA and GQA as the same kind of design with different scales. The granularity of sharing changes a structural property, not just a magnitude; GQA preserves lookup-side diversity that MQA destroys.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.