Encoder Decoder

Where does cross-attention live in OpenAI Whisper?

Inside a T5-style encoder-decoder, which sub-layer reads the encoder outputs, and through which projection?

Translating one source sentence into many target languages, what attention-side trick scales?

Decoder-only LLMs ship without a cross-attention sub-layer, so where does the context go?

Walk through a T5 decoder block in order, list the attention sub-layers and what each one's Q, K, V read from.

Explain when an architecture uses cross-attention vs self-attention. Give concrete examples.

Which modern LLM architecture uses cross-attention?