Zenaique
Part ofFresher / New Grad·Week 4: Interview PrepView roadmap →

Spot the error in this explanation of temperature.

Spot the error·Easy·4.3 · 78·~2 min·Asked atBasetenSiemensYellow Ai·Relevant atOpenAI
Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.
TL;DR

Temperature divides logits before softmax: low T sharpens (deterministic), T=1 leaves the distribution unchanged, high T flattens (more random). The passage gets all three temperature claims wrong.

Memory aid
Sign in to see the mnemonic that makes this stick.
Easy to grasp

Imagine a bag of marbles, one for each word the model could say next. Some marbles are bigger (the model likes those words more) and some are smaller. The model reaches in without looking and pulls one out. Bigger marbles get picked more often. Temperature is a dial. Turn it down low, and the big marbles grow huge while the small ones shrink to almost nothing. You almost always pull the same big marble, so the model says the safe, predictable word every time. Turn it up high, and all the marbles become roughly the same size. Now any word can come out, so the model gets creative and surprising, but sometimes weird. Set it in the middle, and the bag stays exactly as the model packed it.

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Temperature is the most common sampling parameter in LLM APIs, and its direction is easy to get backwards. It divides the logits before softmax: a small T sharpens the distribution toward the top token, and a large T flattens it toward uniform. T=1 leaves the model's own distribution unchanged.

Getting the direction wrong causes real problems. Using T=1 for a path that needs stable JSON leaves the output random, and the schema can break on roughly five percent of requests. Using T=0 for brainstorming returns the same answer every time.

The formula and the direction

The model produces logits z, one per vocabulary token. Softmax turns z into probabilities P:

P(xi)=ezi/T∑jezj/TP(x_i) = \frac{e^{z_i / T}}{\sum_j e^{z_j / T}}

The division by T happens BEFORE the exponential. When T is small (say 0.1), dividing magnifies the differences between logits. The largest logit dominates the softmax, and the resulting distribution concentrates mass on that one token. The sampler nearly always picks the argmax.

When T is large (say 2.0), the divisions shrink the gaps. After exponential and normalisation, all tokens get a meaningful share of probability. The sampler picks more diverse tokens.

When T equals 1, dividing by 1 changes nothing. The softmax sees the raw logits, and you sample from the model's own distribution. Nearly every chat API defaults to it.

The passage inverts all three: T=0 is wrongly called creative (it is greedy), T=1 is wrongly called deterministic (it is unchanged), and low T is wrongly recommended for unpredictable outputs (it is for stable outputs).

Limits at T=0 and large T
Interaction with top-p and top-k
Operational picks across tasks
Default API settings and reproducibility
Sign in to unlock the full deep dive.

Situations where this technique stops working.

Sign in to see when this approach fails.

2–4 min · Everything important, quickly.

Sign in to see the quick scan of the deep dive.

Real products, models, and research that use this idea.

  • OpenAI's Chat Completions API takes a temperature between 0 and 2 and defaults to 1; its reference says lower values such as 0.2 make output more focused and deterministic.
  • Anthropic models released after Claude Opus 4.6 reject a temperature other than 1.0 with a 400 error, so those models cannot be set to T=0.
Sign in to see more production examples.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat's the difference between T=0 and 'greedy decoding'?
A

They are equivalent in practice. T=0 is a limit (division by zero), implemented as argmax. Most frameworks treat T=0 as a special case that skips the softmax and picks the top logit.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Inverting the direction: claiming low temperature is creative and high temperature is deterministic. The opposite is true: low T sharpens the softmax toward argmax.

Sign in to see all red flags and common mistakes.

60 second bullets to scan on the way to the call.

  • The temperature formula: divide logits by T before softmax

  • Direction: low T sharpens, high T flattens

Sign in to unlock the revision sheet.

Primary sources. Browse if you want the original framing.

Similar questions

Same topic, related formats. Practice these next.

4 curated
Next question
What is the KV cache in transformer inference?
Flashcard·Easy