Spot the GQA configuration error: num_attention_heads=32, num_key_value_heads=7.

**Grouped-query attention (GQA)** is a middle ground between full multi-head attention (every query head has its own K and V) and multi-query attention (all query heads share a single K and V). In GQA you pick `n_kv_groups` smaller than `n_heads`. The query heads are split into that many equal groups, and each group shares one K and one V projection. This shrinks the KV cache without sacrificing as much quality as MQA. The configuration above asks for 32 query heads and 7 KV heads. That fails because 32 is not divisible by 7. Each KV head must serve the same number of query heads (the group size must be a whole number), so the only legal choices for `n_kv_heads` are divisors of 32: 1, 2, 4, 8, 16, or 32. Llama-2 70B uses 8, which is almost certainly what the teammate meant. The model does not silently truncate or pad. The attention reshape from `(batch, seq, 32, d_head)` into groups raises a shape mismatch the first time it executes, well before anything reaches staging.

**GQA** needs `num_attention_heads % num_key_value_heads == 0`. The reshape from a flat 32-head tensor into `(n_kv_groups, group_size, d_head)` only works when 32 factors cleanly into the product. 32 mod 7 = 4, so the configuration is broken. The error does not surface in the smoke test the way you might expect. The config file parses fine because both numbers are positive integers. The weights load fine because the K and V projection matrices have shape `(d_model, 7 * d_head)`, which is dimensionally legal on its own. The crash comes at the first forward pass, when the attention kernel tries to broadcast 7 KV heads across 32 query heads and the reshape rejects the non-integer group size. Valid choices for 32 query heads are the divisors: 1 (pure MQA), 2, 4, 8, 16, or 32 (pure MHA). Llama-2 70B settled on 8 as the production sweet spot. The fix is almost certainly to change 7 to 8 in the config. The broader lesson is that 'the model file loaded' is not a meaningful smoke test for shape constraints. A unit test that runs a single forward pass on a batch of two tokens catches GQA misconfigurations in under a second.