**Temperature** is a single decoding knob that controls how the model picks its next token from the probability distribution. At T = 0 the model always picks the highest-probability token, which makes it deterministic. At T > 0 the model samples, so the same prompt can yield different outputs each run.
The choice is not about which is better in the abstract. It is about what the task needs. Tasks with a single right answer (classification, JSON extraction, math) usually want T = 0 because reproducibility and consistency matter more than variety. Tasks that benefit from diversity (creative writing, brainstorming, generating multiple options for a UI) usually want T > 0 so the output is not stuck on one default phrasing.
There is also a structural case for T > 0. Self-consistency voting samples many reasoning paths and votes on the most common answer. That pattern collapses at T = 0 because every sample is identical, so there is nothing to vote on.
Three distinct scenarios favor T > 0 over T = 0 in production. First, tasks that need genuine diversity, such as creative writing, brainstorming, or any UI that surfaces multiple options for the user to pick from. T = 0 forces every output toward the same modal answer, which kills variety.
Second, sampling-based patterns like self-consistency. Self-consistency samples N reasoning paths and votes on the most-common final answer. At T = 0 all N samples are identical because greedy decoding is deterministic, so the vote collapses to one sample and the pattern stops working.
Third, escaping a wrong argmax. Sometimes the model's top-1 token is incorrect even though good answers have meaningful probability mass nearby. T > 0 lets sampling find them; T = 0 locks in the wrong choice every run. Best-of-N over a verifier exploits this directly.
T = 0 wins where the task has a single correct answer and you want reproducibility: classification heads, structured extraction, regression tests in your eval pipeline. The right framing is not which is universally better. It is which property the task needs, determinism or controlled diversity.
Quick reference
TL;DR
T > 0 wins when you need diversity (creative tasks, self-consistency voting) or want to escape a wrong argmax; T = 0 wins for single-answer tasks and reproducibility.
Imagine asking a friend to pick a restaurant. If you always want their single best recommendation, you want them to give the same answer every time. That is temperature equals zero, totally consistent, no variety. But if you want them to brainstorm five different options for a group, you need them to sometimes pick the second or third favorite, not just the absolute top. That is temperature above zero, where the model is allowed to sample less obvious answers. The right setting depends on what the answer is for, not on a universal rule.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Tasks with a single correct answer and a regression-tested eval suite Reproducibility is part of the contract; T = 0 keeps prod and eval bit for bit aligned.
Structured JSON extraction with a strict schema T > 0 adds tokenization noise that produces flaky schemas across runs without any quality upside.
Cost-sensitive single-shot generation with no verifier Without a verifier or vote, sampling just adds variance; T = 0 is more predictable per call.
Low-volume audited workflows where every output is reviewed Reviewers need stable baselines; randomness across re-runs slows audits without helping quality.
Quick scan · 2–4 min
Everything important, quickly.
3 min: softmax with temperature + three T > 0 scenarios + when T = 0 wins + how to pick per task.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic's Claude Opus 4.7 API defaults T = 1 but production extraction pipelines pin T = 0 for reproducible JSON outputs.
2OpenAI's GPT-5.5 docs recommend T = 0 for classification and T = 0.7 to 1.0 for creative generation.
3Cursor's code-completion path uses higher temperature with best-of-N reranking, while its inline JSON tool calls run at T = 0.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does self-consistency specifically need T > 0?
Sign in or go Pro to see the approach for this follow-up.
QHow does top-p interact with temperature?
Sign in or go Pro to see the approach for this follow-up.
QHow would you decide T for a new task in your eval pipeline?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating T = 0 as the safe default for every production task, then being surprised when self-consistency voting collapses because every sample is identical.