Spot the error in this explanation of temperature. · Zenaique

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

Quick reference

TL;DR

Temperature divides logits before softmax: low T sharpens (deterministic), T=1 leaves the distribution unchanged, high T flattens (more random). The passage inverts every claim.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine a bag of marbles, one for each word the model could say next. Some marbles are bigger (the model likes those words more) and some are smaller. The model reaches in without looking and pulls one out. Bigger marbles get picked more often. Temperature is a magic dial. Turn it down low, and the big marbles grow huge while the small ones shrink to almost nothing. You almost always pull the same big marble, so the model says the safe, predictable word every time. Turn it up high, and all the marbles become roughly the same size. Now any word can come out, so the model gets creative and surprising, but sometimes weird. Set it in the middle, and the bag stays exactly as the model packed it.

Key concepts

Temperature sampling Softmax Decoding strategies

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

3 min: softmax with T formula + direction (low sharpens, high flattens) + T=0 versus greedy + interaction with top-p + use cases (code low, creative high).

Real-world examples

Real products, models, and research that use this idea.

OpenAI API defaults to T=1 for chat completions; cookbook recipes for code generation explicitly drop to T=0 or 0.2.
Anthropic Messages API documentation recommends T=0 for deterministic tasks and notes that Claude's calibrated distribution is at T=1.
vLLM and SGLang expose temperature as a per-request sampling parameter alongside top-p and top-k.
GitHub Copilot uses very low temperature plus top-p for code completion to maximise stability.
Hugging Face TGI exposes temperature in the same `SamplingParams` block as top-p, top-k, and repetition penalty.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat's the difference between T=0 and 'greedy decoding'?

Sign in or go Pro to see the approach for this follow-up.

QHow do temperature and top-p interact?

Sign in or go Pro to see the approach for this follow-up.

QWhy does very low but non-zero temperature sometimes hurt quality?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Inverting the direction: claiming low temperature is creative and high temperature is deterministic. The opposite is true, low T sharpens the softmax toward argmax.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.