Multi-Query Attention

Expand MQA and state its KV-cache cost tradeoff

Define GQA in transformer attention

Tying K and V to one projection, what is gained and what is risked?

PremiumWhy does MQA underperform…

MQA, spell out the acronym and name exactly what gets shared across heads

GQA, decode the acronym and describe its KV sharing pattern

Decode phase attention is memory bandwidth bound. Explain what flips the regime.

Match MLA, GQA and MQA to their mechanisms, cache reduction and quality posture

Which statement most accurately captures why GQA, not MQA, became the production default?

Match each attention variant to its KV-cache property and a production user

Compare MHA, MQA, GQA, MLA, what production tradeoff are they all addressing, and which models use each?

PremiumMatch each attention variant…