Explain when an architecture uses cross-attention vs self-attention. Give concrete examples.

The whole difference between self-attention and cross-attention is where Q, K, and V come from. Get that clear and the architectures fall out automatically. In self-attention, all three projections read from the same input x: `q = W_Q x`, `k = W_K x`, `v = W_V x`. Each token attends to other tokens in its own sequence. There are sub-flavors based on the mask, BERT-style bidirectional or GPT-style causal, but the routing is the same. In cross-attention, Q is projected from one source and K, V are projected from a different source. The classic pattern is encoder decoder translation: the encoder reads the source sentence and produces a sequence of hidden states, and the decoder generates the target sentence while cross-attending to those encoder hidden states. So `q = W_Q x_dec`, `k = W_K x_enc`, `v = W_V x_enc`. Each decoder position can attend to all encoder positions. Real architectures map cleanly to families. BERT, RoBERTa, and DeBERTa are encoder-only and use bidirectional self-attention. GPT, Llama, Mistral, and Claude are decoder-only and use causal self-attention; they have no encoder, so no cross-attention. T5, BART, and Whisper are encoder decoder, where each decoder block has three sublayers: causal self-attention, cross-attention to the encoder, then an MLP.

Depends on where Q, K, and V get their input, that one routing fact is the whole difference. In self-attention, all three come from the same sequence x. Q asks, K advertises, V delivers, and the conversation happens within one stream. BERT does this bidirectionally, GPT and Llama do it causally, and the encoder side of T5 does it bidirectionally too. In cross-attention, Q comes from one sequence and K/V come from a different one. The canonical case is the T5 decoder: each block does causal self-attention over the tokens generated so far, then cross-attention with Q from the decoder and K/V from the encoder output, then the MLP. That cross-attention is what lets the decoder ground its generation in the source. One practical consequence worth naming: the cross-attention K/V cache is computed once when the encoder runs and reused across every decode step. The self-attention cache grows by one row per generated token. So encoder decoder inference for fixed input tasks is actually pretty efficient: the expensive cache is only the self-attention one. Decoder-only LLMs skip cross-attention because they prepend everything (instructions, context, query) into one sequence and let causal self-attention handle it all uniformly. That's the architectural simplification that made modern serving stacks work at scale.