Multi-query attention (MQA) and grouped-query attention (GQA) both reduce KV cache by sharing K and V across query heads. Yet MQA usually degrades model quality noticeably while GQA does not. Explain the structural reason for that gap, and what knob makes GQA the safer compression point.
Free · 2 AI evals / day
Both MQA and GQA reduce the KV cache by sharing K and V across query heads. The difference is how much they share, and that difference cascades into a quality gap.
MQA collapses everything: every query head in a layer reads the same K and V. The cache shrinks by a factor of num_heads (the largest possible reduction), but every head now indexes into the same memory. The parallel specialization that multi-head attention is designed to provide is mostly lost on the lookup side.
GQA splits the query heads into G groups and gives each group its own K and V. The cache shrinks by a factor of `num_heads / G` (smaller reduction than MQA but still huge), and each group retains its own K, V subspace. So heads within a group share, but groups remain distinct.
The knob is **group size G**. G=1 is MQA (maximum compression, biggest quality hit). G=num_heads is MHA (no compression, no quality hit). G=8 is the production sweet spot in most 2026 frontier models, capturing most of the cache savings with quality within noise of full MHA.
Production examples: Llama-3 70B and Llama 4 Maverick both use 8 KV heads per 64 query heads.
MQA underperforms GQA because the granularity of K, V sharing determines how much head-level specialization survives.
Multi-head attention's central insight is that different heads can learn to attend to different relations: positional adjacency, syntactic dependencies, coreference, document structure. Each head's specialization is most directly encoded in its (K, V) subspace, the 'shelf' it indexes into. When you collapse to a single shared K, V (MQA), every head reads from the same shelf. Queries can still differ, but every question lands in the same memory.
GQA preserves a partial version of that diversity. With G groups, you have G distinct K, V subspaces per layer. Heads within a group share, but the groups remain genuinely different views of the input. Empirically this is enough to recover most of MHA's quality.
The knob is group size G:
- G=1: MQA. Maximum compression (factor of num_heads), biggest quality hit.
- G=8: production sweet spot. ~8x cache savings (relative to MHA) and quality within noise of MHA.
- G=num_heads: MHA. No compression, no quality cost.
The quality vs G curve is convex: most of the recovery happens between G=1 and G=8, and additional groups beyond that give diminishing returns. That is why production frontier models in 2026 (Llama 4 Maverick, Claude Opus 4.7, Gemini 3.1 Pro, Mistral Large 3) converge on G in the 4 to 8 range.
Quick reference
TL;DR
MQA collapses all heads onto one K, V, killing lookup-side diversity. GQA preserves group-level K, V diversity via a group-size knob, keeping most quality at almost MQA's cache size.
Imagine a research team of 64 analysts. Multi-head attention gives each analyst their own personal library. MQA fires the librarians and forces all 64 analysts to share one library, much cheaper, but every analyst now looks at the same shelf, so their analyses become similar. GQA splits the analysts into 8 groups of 8 and gives each group its own library, almost as cheap as one shared library but with 8 different views into the source material. The group size is the knob.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Memory budgets that force MQA's maximum compression If MQA is the only way to fit the deployment, the quality hit may be acceptable; GQA's sweet spot only matters when you have room to choose.
Models with very small num_heads When num_heads is small (say 8 or fewer), there are few groups to choose from, so GQA's flexibility is limited and the MQA vs GQA gap shrinks.
Architectures using MLA latent compression MLA replaces the per-head K, V structure entirely with a compressed latent; the GQA vs MQA comparison does not directly apply.
Tasks dominated by short-context single-turn quality Cache savings matter most at long context; if your deployment is short-context dominated, MHA or large-G GQA may be more appropriate.
Quick scan · 2–4 min
Everything important, quickly.
5m: why MHA's head diversity matters, how MQA's single shared K, V collapses lookup-side diversity, what GQA's group size G preserves, the convex quality vs G curve, and the production convergence on G in the 4 to 8 range.
Side-by-side
Property
MQA
GQA (G=8)
MHA
KV heads per layer
1
8
num_heads
Cache reduction vs MHA
num_heads x
num_heads / 8 x
1x (baseline)
Lookup-side diversity
None
8 subspaces
num_heads subspaces
Quality vs MHA
Measurable drop
Within noise
Baseline
Production usage in 2026
Rare (legacy)
Default
Encoders, small models
Real-world examples
Real products, models, and research that use this idea.
1Llama-3 70B uses GQA with 8 KV heads per 64 query heads.
2Llama 4 Maverick continues the G=8 pattern as the production default.
3Claude Opus 4.7 and Gemini 3.1 Pro both use GQA at similar group counts.
4Mistral Large 3 uses GQA combined with a sliding-window and full-attention layer hybrid.
5The Ainslie et al. 2023 GQA paper empirically swept group sizes from 1 (MQA) to num_heads (MHA) and showed the convex quality curve.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is the quality vs G curve convex rather than linear?
Sign in or go Pro to see the approach for this follow-up.
QHow would you choose G for a deployment with a fixed memory budget?
Sign in or go Pro to see the approach for this follow-up.
QHow does MLA in DeepSeek V4 sidestep the MQA vs GQA tradeoff?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating MQA and GQA as the same kind of design with different scales. The granularity of sharing changes a structural property, not just a magnitude; GQA preserves lookup-side diversity that MQA destroys.