Click any words you think contain an error. Click again to unmark.
Temperature divides logits before softmax: low T sharpens (deterministic), T=1 leaves the distribution unchanged, high T flattens (more random). The passage gets all three temperature claims wrong.
Imagine a bag of marbles, one for each word the model could say next. Some marbles are bigger (the model likes those words more) and some are smaller. The model reaches in without looking and pulls one out. Bigger marbles get picked more often. Temperature is a dial. Turn it down low, and the big marbles grow huge while the small ones shrink to almost nothing. You almost always pull the same big marble, so the model says the safe, predictable word every time. Turn it up high, and all the marbles become roughly the same size. Now any word can come out, so the model gets creative and surprising, but sometimes weird. Set it in the middle, and the bag stays exactly as the model packed it.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Temperature is the most common sampling parameter in LLM APIs, and its direction is easy to get backwards. It divides the logits before softmax: a small T sharpens the distribution toward the top token, and a large T flattens it toward uniform. T=1 leaves the model's own distribution unchanged.
Getting the direction wrong causes real problems. Using T=1 for a path that needs stable JSON leaves the output random, and the schema can break on roughly five percent of requests. Using T=0 for brainstorming returns the same answer every time.
The formula and the direction
The model produces logits z, one per vocabulary token. Softmax turns z into probabilities P:
The division by T happens BEFORE the exponential. When T is small (say 0.1), dividing magnifies the differences between logits. The largest logit dominates the softmax, and the resulting distribution concentrates mass on that one token. The sampler nearly always picks the argmax.
When T is large (say 2.0), the divisions shrink the gaps. After exponential and normalisation, all tokens get a meaningful share of probability. The sampler picks more diverse tokens.
When T equals 1, dividing by 1 changes nothing. The softmax sees the raw logits, and you sample from the model's own distribution. Nearly every chat API defaults to it.
The passage inverts all three: T=0 is wrongly called creative (it is greedy), T=1 is wrongly called deterministic (it is unchanged), and low T is wrongly recommended for unpredictable outputs (it is for stable outputs).
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI's Chat Completions API takes a temperature between 0 and 2 and defaults to 1. Its reference says lower values such as 0.2 make output more focused and deterministic.
- Anthropic models released after Claude Opus 4.6 reject a temperature other than 1.0 with a 400 error, so those models cannot be set to T=0.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat's the difference between T=0 and 'greedy decoding'?
They are equivalent in practice. T=0 is a limit (division by zero), implemented as argmax. Most frameworks treat T=0 as a special case that skips the softmax and picks the top logit.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Inverting the direction: claiming low temperature is creative and high temperature is deterministic. The opposite is true: low T sharpens the softmax toward argmax.
60 second bullets to scan on the way to the call.
The temperature formula: divide logits by T before softmax
Direction: low T sharpens, high T flattens
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.