Mha

Llama-2 70B uses 64 query heads, how many KV heads does it actually keep?

GQA, decode the acronym and describe its KV sharing pattern

Which statement most accurately captures why GQA, not MQA, became the production default?

Match each attention variant to its KV-cache property and a production user

Compare MHA, MQA, GQA, MLA, what production tradeoff are they all addressing, and which models use each?

PremiumMatch each attention variant…