Llama-2 70B uses 64 query heads, how many KV heads does it actually keep?

Attempt it

Quick reference

TL;DR

Llama-2 70B uses 8 KV heads for 64 query heads, an 8x KV cache reduction via grouped-query attention. The 7B and 13B use plain MHA.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine 64 students all asking different questions in a library. Plain multi-head attention assigns 64 personal librarians and 64 bookshelves, very expensive at scale. Llama-2 70B groups the students into 8 study rooms of 8 each; everyone in one room shares one librarian and one shelf. The questions stay individual (each student keeps their own query), but the reference material is shared. You go from 64 librarians to 8, and the students' results stay nearly as good. That sharing pattern is what GQA does for the KV cache.

Key concepts

Grouped-query attention KV cache Llama-2 70B Multi-head attention

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

60s: state the numbers (64 query heads, 8 KV heads, group size 8), explain that it is GQA, give the KV cache reduction (8x: 21 GB to 2.7 GB at 8k context), note that 7B and 13B kept plain MHA, mention G = 8 was on the flat part of the quality curve in Meta's ablations.

Real-world examples

Real products, models, and research that use this idea.

Llama-2 70B config.json: num_attention_heads = 64, num_key_value_heads = 8. The two-field exposure of GQA in the Hugging Face spec.
Llama 4 Maverick continues the GQA pattern with similar query-to-KV head ratios at its frontier scale.
Mistral Large 3 uses GQA across its model family for the same KV cache motivation.
Qwen 3.5 series tunes group sizes per model size, with the larger variants leaning into deeper sharing.
Gemma 4 uses GQA across its model family with model size specific KV head counts.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy did Meta pick GQA-8 rather than MQA for the 70B?

Sign in or go Pro to see the approach for this follow-up.

QHow does Llama-2's GQA-8 choice translate to inference batch size on an H100?

Sign in or go Pro to see the approach for this follow-up.

QWhat does Llama 3 70B / Llama 4 Maverick do differently on this axis?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Assuming every Llama-2 variant uses GQA. Only the 70B does in the original release. The 7B and 13B kept plain MHA where the KV cache was not the binding constraint.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.