By 2020 T5-style encoder-decoders led on most seq2seq benchmarks. By 2024 every frontier general purpose LLM was decoder-only. Explain what changed.
Free · 2 AI evals / day
**Three things changed between 2020 and 2024.**
First, **scale**. A sufficiently large decoder-only model can match a smaller specialized encoder-decoder on the same task, just by being given a task prefix like 'translate to French:'. GPT-3 (175B parameters) demonstrated this in 2020; by GPT-4 era it was overwhelming.
Second, **instruction tuning**. Decoder-only models can be trained on a mixture of pretraining data and instruction-following examples using one objective (predict the next token). Encoder-decoders need explicit parallel input/output pairs and a separate cross-attention path. Instruction tuning turned a general purpose decoder-only model into a system that can do any task you ask it to.
Third, **simplicity compounds**. One block type is easier to scale, easier to optimize, easier to serve, and uses one KV cache structure for every workload. Encoder-decoders need separate code paths for the encoder, the decoder, and the cross-attention.
**Encoder-decoders still win in niches.** Whisper uses the encoder once to process audio, then the decoder generates text while cross-attending to the cached encoder output. That reuse is hard to replicate in a decoder-only model.
**The decoder-only architecture won because scale + instruction tuning made the encoder-decoder split unnecessary for general purpose work.** Three causes.
**Cause 1: scale subsumes architectural specialization.** In 2020, T5 base (220M params) was strictly better at translation than GPT-2 (1.5B) was. By 2022, GPT-3 (175B) with a 'translate to French:' prefix matched T5-XXL (11B) on most translation benchmarks. Scale collapsed the architectural advantage. The same pattern played out on summarization, question answering, and code generation.
**Cause 2: instruction tuning unifies the objective.** Decoder-only models train on next-token prediction across mixed data. Once you can frame any task as 'instruction: \<task\>, input: \<x\>, output: \<y\>' in a single autoregressive sequence, you can train one model on hundreds of tasks at once. Encoder-decoders require explicit (input, output) parallel pairs at training time, which is more restrictive.
**Cause 3: architectural simplicity compounds.** One block type means one inference kernel, one KV cache layout, one serving stack. The KV cache trick that makes decoder-only inference fast at long context is uniform across every workload. Encoder-decoder serving requires separate code paths and the cross-attention KV is shaped differently from self-attention KV.
**Where encoder-decoder still wins.** Whisper for speech to text (the encoder reads mel spectrograms once, the decoder reuses that representation across every output token). NLLB for high throughput translation across many language pairs. Any task where the input modality differs structurally from the output, or where the same input drives many output generations, still favors encoder-decoder.
Quick reference
TL;DR
Scale plus instruction tuning let one decoder-only model do every task with a prefix; the architectural split between encoder and decoder became unnecessary overhead.
In 2020, the best way to translate French was a specialized model with two halves: one half read French, the other half wrote English. Each half had a different job. Then GPT-3 got large and people noticed something surprising: if you just typed 'translate this French to English:' and pasted the French, a single half model could do it almost as well. As models got bigger and learned to follow instructions, the specialized two half design lost its advantage. A general model with a prompt prefix could do translation, summarization, question answering, and code generation, all from the same trained model. The two half design only survives in niches where the input is a totally different kind of data (like audio) or where the same input drives many outputs.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Building a production speech to text system Encoder-decoder is still the right choice; Whisper's encoder runs once on audio and the decoder reuses that representation across every output token.
Translating one source into 50 target languages at high throughput Encoder-decoder amortizes the encoder pass across all target languages; decoder-only would re-process the source for every target.
Designing a tiny on device model under 100M parameters At very small scale, encoder-decoder's architectural specialization still pays off; decoder-only's advantage only kicks in past ~1B parameters.
Quick scan · 2–4 min
Everything important, quickly.
6 min: name the three causes, walk the GPT-3 vs T5 comparison, explain instruction tuning's role, cover the serving simplicity story, and finish with the Whisper / NLLB niches.
Side-by-side
Property
Encoder-decoder (T5-style)
Decoder-only (GPT-style)
Best for
Fixed input, variable output, many outputs per input
General purpose, instruction following, chat
Training objective
Span corruption or denoising on parallel (input, output) pairs
Real products, models, and research that use this idea.
1GPT-5.5, Claude Opus 4.7, Gemini 3.1 Pro, Llama 4 Maverick: every frontier 2026 general purpose LLM is decoder-only.
2Whisper (OpenAI): still encoder-decoder for speech to text because the audio encoder representation is reused across every output token.
3NLLB (Meta): encoder-decoder for batch translation because one encoder pass amortizes across many target language decoder runs.
4DeepMind's Gemini family ships as decoder-only despite being multimodal; the multimodal inputs are tokenized into the same autoregressive stream rather than being routed through a separate encoder.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is 'prefix-LM' and how does it sit between encoder-decoder and decoder-only?
Sign in or go Pro to see the approach for this follow-up.
QCould encoder-decoder make a comeback for multimodal models?
Sign in or go Pro to see the approach for this follow-up.
QWhy didn't BERT's encoder-only architecture win for any modern frontier task?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Saying 'decoder-only is simpler so it won'. Simplicity matters but scale + instruction tuning is the real cause. T5 didn't lose because it was complex; it lost because GPT-3-scale models with prefixes matched it on the actual task.