Encoder-Decoder
Also known as: Encoder decoder, Seq2seq transformer
Transformer with separate encoder + decoder stacks; strong for translation and structured seq2seq tasks.
A transformer architecture with separate encoder and decoder stacks. The encoder builds a contextual representation of the input; the decoder generates output tokens attending to both the encoder output and prior decoded tokens. Used by T5, BART, original Transformer.
In practice
Contrast point for decoder-only models. Senior architecture interviews probe when each shape is the right call (translation vs chat).
How it compares
Encoder-decoder has separate stacks with cross-attention; decoder-only uses one stack and prefix-feeds input as context.
Comparisons that include Encoder-Decoder
Related topics
Questions that mention this term
- Identify the attention side optimization when translating one source sentence into many target languages
- Explain the difference between self-attention and cross-attention. When does each appear in a real architecture?
- Why did decoder-only architectures…
- Name a task where an encoder-decoder transformer still beats a decoder-only LLM
- Order the attention sub-layers in a T5 decoder block and what each one's Q, K, and V read from in self-attention versus in cross-attention.
- Explain how decoder-only LLMs condition on context without a cross-attention sub-layer
Related terms
Attention Mechanism
How a model decides which input tokens to weight when computing each output token.
Causal Mask
Attention mask that hides future tokens so each position can only see itself and prior tokens.
Context Window
The max number of tokens a model can attend to at once.
Decoder-Only
Single autoregressive transformer stack: the shape of every modern frontier LLM.
Layer Normalization
Per-sample, per-position normalization. The glue that keeps transformer training stable.
Mixture of Experts (MoE)
Scale model capacity by routing each token to a small subset of expert MLPs instead of using all of them.