Define GQA in transformer attention

**GQA** stands for **Grouped-Query Attention**. In a normal multi-head attention layer, every query head has its own matching key head and value head. If a model has 32 Q heads, it also has 32 K heads and 32 V heads, and every one of those K and V tensors has to live in the KV cache during decoding. GQA changes the bookkeeping. The Q heads are split into G groups, often 8 groups. Each group still has its own queries, but the whole group shares a single K head and a single V head. So 32 Q heads paired with 8 KV groups means 8 K tensors and 8 V tensors per layer instead of 32 of each. The shared KV tier is what shrinks the KV cache. Smaller cache means more tokens fit in memory, longer contexts fit on the same GPU, and decode reads less from HBM each step. Modern open-weights models like Llama 3.1, Mistral, and Qwen 3.5 all ship with GQA at group size 8.

**GQA**, or Grouped-Query Attention, sits between full multi-head attention and the more aggressive multi-query variant. Q heads keep their individuality, but they are partitioned into G groups, and every head inside a group shares one K head and one V head with its peers. The motivation is decode-time bandwidth. Decoding a token reads the entire KV cache from HBM. With multi-head attention, every Q head pulls its own K and V slab, which dominates the cost. Collapsing the K and V side to G shared heads cuts that read by `n_heads / G`, typically a 4x to 8x reduction at group size 8. The Q side is untouched, so representational capacity stays high. Quality lands within a fraction of a percent of full multi-head attention on standard evals, well above what pure multi-query attention delivers. Llama 3.1, Mistral, and Qwen 3.5 all adopted GQA at group size 8. GQA's headline win is therefore a smaller KV cache, faster decode, and longer context windows on the same GPU, without the quality hit MQA pays.