Temperature
Also known as: Sampling temperature
Sampling knob (low = focused, high = diverse), applied to the logits before softmax.
A sampling parameter that controls the randomness of LLM outputs. Low temperature (0.0-0.3) produces deterministic, focused responses; high temperature (0.7-1.0+) produces more creative, diverse outputs.
In practice
First knob to reach for when output is too random or too repetitive. Expect questions on its interaction with top-p and top-k.
Related topics
Practice questions
Questions that mention this term
- Dividing pre-softmax attention scores by an extra factor > 1 at inference does what?
- Which of the following are components of a production grade prompt beyond 'just instructions'?
- Select the techniques that genuinely cut hallucination in a RAG answer
- What is the primary reason for running LLM judges at temperature=0 in an eval pipeline?
- Greedy decoding and temperature sampling: does one cost more per token than the other?
- Compare top_k and top_p as truncation strategies for sampling
Related terms
API LLM
An LLM accessed through a provider API: pay per token, get the frontier model, hand over ops.
Beam Search
Keep the K best partial sequences at each step; deterministic, breadth-first decoding.
FlashAttention
A memory-aware attention kernel that's 2-4x faster than vanilla, with identical math.
GGUF
Self-contained binary format for quantized LLMs; the standard for llama.cpp / Ollama / LM Studio.
Greedy Decoding
At each step, pick the single highest-probability token. Fast and deterministic, but often loops.
Grouped-Query Attention (GQA)
Compromise between MHA and MQA: query heads share KV heads in groups, cutting KV cache by 4-8x.