GQA gives a tunable knob: share K and V across groups of query heads to shrink the KV cache like MQA, while keeping enough KV heads to hold MHA-level quality.
Picture a classroom where every student (a query head) needs to consult reference notes before answering. Multi-head attention gives each student a personal, hand-written copy of the notes, which is accurate but eats a huge amount of paper. Multi-query attention hands the whole class a single shared copy, saving paper but causing crowding and mistakes when the material gets hard. Grouped-query attention splits the class into small groups, say eight students per shared copy. You still save most of the paper, but each group's notes stay specialized enough that answers barely suffer. That middle setting is why modern language models pick it: nearly the savings of one shared copy, nearly the quality of personal copies.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
This question looks like a trivia comparison, but it is really probing whether you understand what dominates the cost of language-model decoding. The honest answer is the KV cache, and every attention variant in this question is a different bet on how much of that cache you can throw away before quality suffers. An interviewer who asks this is not testing whether you have memorized three acronyms. They want to know whether you can name the resource being optimized and explain why the obvious-sounding alternatives are red herrings.
Multi-head attention, multi-query attention, and grouped-query attention all compute attention the same way. They differ only in how many distinct key and value heads they keep. That single design choice changes the size of the KV cache, which in turn changes how many bytes the GPU must stream from high-bandwidth memory on every decode step. Because decode is bandwidth-bound rather than compute-bound, shrinking the cache directly raises throughput. A model that halves its cache can roughly double the number of concurrent requests it serves on the same hardware, which translates directly into cost per token.
The trap in the wrong options is that they all sound plausible. They invoke compute cost, parameter count, and cache direction, which are exactly the things a candidate reaches for when they have not internalized that this is a memory-bandwidth problem. The correct answer is the only one that names the right axis and the mechanism: a tunable group size that recovers MHA quality while delivering the cache reduction MQA promised. This deep dive walks through each variant, the formula that ties them together, why the field converged on a group size near eight, how an existing model can be cheaply converted, and how newer compression schemes extend the same idea.
The three variants, defined by KV head count
Start from multi-head attention. With n_heads query heads, MHA also maintains n_heads key heads and n_heads value heads. Each query head attends through its own private key and value, so every head can learn to look for a different kind of relationship. Maximum expressiveness, maximum cache.
Multi-query attention keeps all the query heads but collapses to a single key head and a single value head shared across every query. The query side is unchanged, so the model still has the same number of attention computations, but the cache now stores keys and values for one head instead of many. All sixty-four query heads in a typical model now read from the same key, which is where the capacity pressure comes from.
Grouped-query attention generalizes both. You pick an intermediate KV head count. Each KV head is shared by a contiguous group of query heads, so a group of eight query heads might share one key and one value head. Set the KV head count to one and you have MQA. Set it equal to n_heads and you have MHA. Everything in between is a valid GQA configuration, which is exactly the tunability the correct answer points at. The single integer that picks the operating point is what makes the family a continuum rather than three separate designs.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Llama 3.1 70B ships GQA with eight KV heads, the canonical production setting that balances cache size against quality.
- Mistral Large 3 uses grouped-query attention so long-context decode stays within HBM budgets on a single node.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does sharing KV heads barely change compute even though it cuts memory?
Trace the matmuls. Every query head still computes its own scores against the shared key, so the number of query times key dot products is unchanged. What shrinks is the size of the K and V tensors held in HBM and streamed each decode step.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Claiming GQA wins on compute or parameter count. The win is KV-cache bytes and bandwidth; the query side matmul is unchanged, and only the K and V projections shrink.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.