Attention has three projections from the input: Q, K, and V. Q is what the current token is asking for. K is the lookup key, the thing that gets matched against queries. V is the content delivered when a match happens.
Why does V need its own projection instead of just reusing K? Because K and V have different jobs.
K is optimized for matchability. The model wants K vectors that are easy to align with relevant queries via the dot product. Two tokens with similar meanings should have similar K vectors so the same query retrieves both.
V is optimized for delivery. Once a token is selected by the attention weights, its V vector gets summed into the output. The model wants V to carry useful content for the downstream FFN to process. V vectors care about being informative, not about being matchable.
These two objectives are different. A good K is one that's findable; a good V is one that's informative. Forcing them to be the same vector (by tying K = V) means one tensor has to satisfy both objectives simultaneously, and it ends up being mediocre at both.
The analogy: in a library, the title makes a book findable, the body of the book is what you read. Forcing the body to equal the title destroys the library.
**K and V have different jobs.** K is the lookup key, optimized to be matchable against queries. V is the content payload, optimized to be useful when delivered into the weighted sum. Two different optimization objectives, two different optimal representations, two separate projections.
The mechanics make this concrete. K appears in the softmax via `softmax(Q K^T / sqrt(d_k))`. The gradient flowing back to K pushes it toward being a good 'address': similar to queries that should retrieve this token, dissimilar from queries that shouldn't. V appears only in the weighted sum: `output_i = sum_j a_ij V_j`. The gradient flowing back to V pushes it toward carrying features that the next layer's FFN can productively use.
These two gradient signals pull in different directions. K wants discriminability against query vectors; V wants semantic richness for downstream processing. Letting K and V live in separate projections gives each the freedom to specialize for its own objective.
Empirically, models with tied K = V underperform standard attention by 1 to 3 percent perplexity. Frontier models almost universally keep K and V as distinct projections (then share across heads via MQA, GQA, or MLA, but that's a different kind of sharing). The wrong answers in this question reflect common misconceptions:
- **Parameter regularization**: no, more parameters generally help; separation is about roles, not regularization.
- **Memory layout**: no, the kernels handle layout regardless.
- **Non-linearity**: no, V is a pure linear projection.
Quick reference
TL;DR
K is shaped to be findable by queries; V is shaped to be useful content delivered after the match. Two different jobs, two different optimal representations, two separate projections.
Picture browsing a library. A book's title makes it findable on the shelf, but the title is not what you actually read once you pull the book down. The body of the book is what you came for. If every book had to use its title text as its entire content, you'd have a library where the only thing inside any book is its own name repeated. Useless. Real libraries (and real transformers) keep titles separate from contents so each can be designed for its own purpose, finding versus reading.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Extremely memory-constrained on-device transformers where every byte matters more than every perplexity point Tying K = V halves the projection parameters and the KV-cache footprint; the quality cost may be acceptable when fitting the model is the binary constraint.
Models specifically targeting research into attention compression or weight sharing Tied K = V is a useful ablation baseline for studying which structural choices matter most, even if it's not the right production choice.
Tasks where the lookup and the content are intrinsically the same (highly local, redundant context) If the application never requires finding a token by an attribute different from its content (some narrow signal-processing tasks), tying K = V may not hurt as much; this is rare in language modeling.
Architectures that don't use dot-product attention (linear attention, kernel attention, state-space models) The K vs V distinction is specific to the softmax-attention formulation; other architectures have different decompositions of the lookup vs delivery roles.
Quick scan · 2–4 min
Everything important, quickly.
4m: K's role as lookup key, V's role as content payload, the different gradient signals shaping each, why tying K = V hurts quality empirically, and how this differs from head-level sharing in MQA / GQA.
Real-world examples
Real products, models, and research that use this idea.
1Every production transformer since the original Vaswani 2017 paper keeps K and V as separate projections; the design has not been seriously revisited at frontier scale.
2Llama 4 Maverick uses GQA: K and V are shared across query heads (8 KV heads for 64 query heads) but K and V remain distinct tensors per group.
3Claude Opus 4.7 and Gemini 3.1 Pro both use GQA with the same K-vs-V separation.
4DeepSeek V4's MLA compresses K and V into a shared low-rank latent, but decompresses them into distinct K and V tensors before the attention computation.
5Research ablations (DeLighT, several attention-efficiency papers) consistently show 1 to 3 percent perplexity regression when K and V are tied to the same projection.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow is sharing K and V across heads (MQA, GQA) different from tying K to V within a head?
Sign in or go Pro to see the approach for this follow-up.
QWhat would happen if you trained V with a non-linear projection (V = relu(xW_V))?
Sign in or go Pro to see the approach for this follow-up.
QWhy doesn't the model just learn to make K = V automatically if they're useful as the same vector?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Believing the V projection adds non-linearity. It doesn't, V is a pure linear projection of the input. The reason for a separate V is functional separation, not non-linearity.