Self Attention

Encoder self-attention versus decoder self-attention, what is the single structural difference at the mask level?

Inside a T5 style encoder-decoder, which sub-layer reads the encoder outputs, and through which projection?

Walk through a T5 decoder block in order, list the attention sub-layers and what each one's Q, K, V read from.

Explain when an architecture uses cross-attention vs self-attention. Give concrete examples.

PremiumWhich modern LLM architecture…

In self-attention, where do Q, K, and V come from? In cross-attention?