Cross Attention

Where does cross-attention live in OpenAI Whisper?

Fill the blank: for a single head, QK^T has shape (T_query, ___).

Match each transformer family to the kind of attention it uses

Inside a T5 style encoder-decoder, which sub-layer reads the encoder outputs, and through which projection?

Translating one source sentence into many target languages, what attention side trick scales?

Decoder-only LLMs ship without a cross-attention sub-layer, so where does the context go?

Walk through a T5 decoder block in order, list the attention sub-layers and what each one's Q, K, V read from.

Critique this claim about CLIP using cross-attention between towers

Explain when an architecture uses cross-attention vs self-attention. Give concrete examples.

PremiumWhich modern LLM architecture…

In self-attention, where do Q, K, and V come from? In cross-attention?