Grouped-Query Attention

Which recipe does a frontier open weight LLM block ship?

At batch 16 and 4k context with FP8 weights, which model still fits on one 80 GB H100?

Per token decode time grows linearly as a chat session lengthens, debug it

Expand MQA and state its KV-cache cost tradeoff

Define GQA in transformer attention

What do you gain and lose by tying K and V to one projection?

Spot the GQA configuration error: num_attention_heads=32, num_key_value_heads=7.

Which config token names the count of parallel attention heads in a layer?

PremiumWhy does MQA underperform…

How many KV heads does Llama-2 70B keep with 64 query heads?

What does GQA stand for, and describe its KV sharing pattern?

Match GPT-2 versus Llama-2 attention design choices to their differences

Why is decode phase attention memory bandwidth bound when the model generates one token at a time?

How does DeepSeek's Multi-head Latent Attention (MLA) compress the KV cache below GQA?

Match MLA, GQA and MQA to their mechanisms, cache reduction and quality posture

Compute the KV cache size in GB for given model dimensions

Which statement most accurately captures why GQA, not MQA, became the production default?

Match each attention variant to its KV-cache property and a production user

For a 64-Q-head model with GQA group size G=8, how many K/V heads exist and what is the cache reduction vs MHA?

Compare MHA, MQA, GQA, and MLA. What production tradeoff does each one address, and which models use it?

PremiumMatch each attention variant…

PremiumWhy is KV cache…

Compute the KV cache memory for a single request at 4096 context on Llama-2 70B (MHA) in FP16.