Greedy Decoding
Also known as: Argmax decoding
At each step, pick the single highest-probability token. Fast and deterministic, but often loops.
The simplest decoding strategy: at each step, pick the single highest-probability next token. Fast and deterministic but prone to repetitive loops and missed globally-better completions.
In practice
The simplest baseline. Knowing why everyone uses temperature/top-p instead is foundational.
How it compares
Greedy picks the argmax; top-p samples from a dynamic nucleus of plausible tokens.
Comparisons that include Greedy Decoding
Related topics
Questions that mention this term
- What is the primary reason for running LLM judges at temperature=0 in an eval pipeline?
- Greedy decoding and temperature sampling: does one cost more per token than the other?
- Why do production chat APIs almost never offer beam search as a decoding option?
- Flashcard: what does the temperature parameter do during LLM generation and how should you set it?
- How does the temperature parameter reshape the sampling distribution?
- Pair each decoding strategy with its determinism and diversity profile.
Related terms
API LLM
An LLM accessed through a provider API: pay per token, get the frontier model, hand over ops.
Beam Search
Keep the K best partial sequences at each step; deterministic, breadth-first decoding.
FlashAttention
A memory-aware attention kernel that's 2-4x faster than vanilla, with identical math.
GGUF
Self-contained binary format for quantized LLMs; the standard for llama.cpp / Ollama / LM Studio.
Grouped-Query Attention (GQA)
Compromise between MHA and MQA: query heads share KV heads in groups, cutting KV cache by 4-8x.
Knowledge Distillation
Train a small student model to match a big teacher's outputs: cheap, fast inference with most of the quality.