Encoder-Decoder
Also known as: Encoder decoder, Seq2seq transformer
Transformer with separate encoder + decoder stacks; strong for translation and structured seq2seq tasks.
A transformer architecture with separate encoder and decoder stacks. The encoder builds a contextual representation of the input; the decoder generates output tokens attending to both the encoder output and prior decoded tokens. Used by T5, BART, original Transformer.
In practice
Contrast point for decoder-only models. Senior architecture interviews probe when each shape is the right call (translation vs chat).
How it compares
Encoder-decoder has separate stacks with cross-attention; decoder-only uses one stack and prefix-feeds input as context.
Comparisons that include Encoder-Decoder
Related topics
Related terms
Attention Mechanism
How a model decides which input tokens to weight when computing each output token.
Transformer
The attention-only neural architecture behind GPT, Claude, Gemini, and almost every modern LLM.
Context Window
The max number of tokens a model can attend to at once.
Mixture of Experts (MoE)
Scale model capacity by routing each token to a small subset of expert MLPs instead of using all of them.
RoPE (Rotary Position Embedding)
Position info injected by rotating Q and K vectors, easy to extend to longer contexts.
Multi-Head Attention (MHA)
Run several attention heads in parallel with different projections, then concat. This captures multiple relationship types per layer.