Cross-attention is one of two flavors of attention in transformers, and which flavor a model uses depends entirely on the architectural family.
Self-attention is when Q, K, and V are all produced from the same input sequence. Every modern transformer layer has at least one self-attention block. Cross-attention is when Q comes from one sequence and K and V come from a different sequence. The canonical use case is translation: an encoder reads the source language and a decoder generates the target language while attending back to the encoder's output.
The three architectural families to remember are:
- Encoder decoder: T5, BART, the original transformer, Whisper. Has both encoder and decoder, and the decoder uses cross-attention.
- Encoder-only: BERT, RoBERTa, DeBERTa. Bidirectional self-attention. No decoder, no cross-attention.
- Decoder-only: GPT family, Llama, Mistral, Claude. Causal self-attention. No encoder, no cross-attention.
So the answer is T5 and BART. The famous chat LLMs are decoder-only and do not use cross-attention at all.
Cross-attention lives in the decoder of encoder decoder models, which means T5 and BART among the options here.
In an encoder decoder model, the encoder processes the source sequence into a stack of hidden states. The decoder then has cross-attention layers where Q is projected from the decoder's own residual stream and K, V are projected from the frozen encoder output. That cross-attention is what lets the decoder ground its generation in the source.
The other options are wrong for a specific reason each. Llama is decoder-only: no encoder at all, so no cross-attention. BERT is encoder-only, all bidirectional self-attention. And the claim that 'all modern transformers use both' is just false: every modern chat LLM since GPT-3 has been decoder-only, with no cross-attention anywhere in the stack.
One trade off worth naming: the field largely abandoned cross-attention for general LLMs because decoder-only architectures are simpler to scale and serve. You prepend everything into one sequence and let causal self-attention handle context: no separate encoder pass, no two tower architecture. Cross-attention is still alive in multimodal models like Flamingo and in speech to text encoder decoders like Whisper, but for pure-text chat models, it's gone.
Quick reference
TL;DR
T5 and BART, the encoder decoder family, use cross-attention in the decoder. Decoder-only LLMs like GPT, Llama, and Mistral have no encoder, so they use self-attention only.
Think of two ways to write. The first is jotting in a notebook: you only ever look at what you've already written, one stream, one author. That's self-attention. The second is a translator at a desk with a foreign manuscript on one side and a blank page on the other. While drafting on the blank page, the translator keeps glancing across to the source. That cross-referencing between two separate documents is cross-attention. Decoder-only LLMs like GPT and Llama are the notebook writer, only one stream. T5 and BART are the translator at the desk, with a separate encoder stream to glance over.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Building a general-purpose chat LLM in 2026 The decoder-only paradigm dominates because single-stack serving (KV cache, paged attention, prefill/decode split) is much simpler than maintaining a separate encoder pass.
Streaming generation where the input is also growing token by token Cross-attention assumes the encoder output is fixed before decoding starts; a continuously-growing input has no natural place to encode, so causal self-attention over the joined stream is the right primitive.
Latency-critical single-turn completions on short prompts The two-stack encoder decoder pass costs an extra forward over the source; for short prompts on a decoder-only model, in context prepending is faster end to end.
In context learning workflows that mix instructions, examples, and queries Decoder-only architectures handle this by concatenation, which is more flexible than designing an encoder schema for what counts as 'input' vs 'output'.
Quick scan · 2–4 min
Everything important, quickly.
Define the Q/K/V routing distinction, walk the three architectural families with canonical examples, explain why the field moved to decoder-only for general LLMs, and show where cross-attention is still essential today (multimodal, speech).
Side-by-side
Architecture
Attention types used
Cross-attention
Examples
Encoder decoder
Bidirectional self + causal self + cross
Yes, decoder side
T5, BART, Whisper, original Transformer
Encoder-only
Bidirectional self
No
BERT, RoBERTa, DeBERTa
Decoder-only
Causal self
No
GPT-5.5, Claude Opus 4.7, Llama 4 Maverick, Gemini 3.1 Pro
Real-world examples
Real products, models, and research that use this idea.
1T5: every decoder block has causal self-attention, then cross-attention to the encoder output, then MLP.
2BART: denoising encoder decoder used for summarization and structured generation.
3Whisper: encoder decoder for speech to text; the text decoder cross-attends to the audio encoder's mel spectrogram representation.
4Flamingo: gated cross-attention layers injected into a frozen LM to bring in vision features without retraining the language backbone.
5GPT-5.5, Claude Opus 4.7, Llama 4 Maverick, Gemini 3.1 Pro, DeepSeek V4: all decoder-only with no cross-attention anywhere in the stack.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy did the field shift from encoder decoder to decoder-only for general LLMs?
Sign in or go Pro to see the approach for this follow-up.
QHow does Flamingo's gated cross-attention work and why is gating necessary?
Sign in or go Pro to see the approach for this follow-up.
QWhy does the cross-attention KV cache only need to be computed once per inference?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Assuming every transformer has cross-attention because the original 2017 paper did. Decoder-only architectures dominate modern LLMs and never use it.