Self-consistency votes across N distinct reasoning paths. At T = 0 the model is deterministic, all N samples are identical, and the vote collapses to a single sample.
Imagine asking ten people the same hard riddle and going with whichever answer the most people gave. That works because each person reasons differently and you get a real majority. Now imagine ten clones of the same person, all thinking in exactly the same way. They will give the same answer every time. There is no real vote, just one opinion repeated. Self-consistency is the first situation, and it needs the model to think a little differently each run. Temperature equals zero turns it into the second situation, with ten identical clones.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Self-consistency is interesting because its entire value comes from diversity. Strip the diversity and the pattern still runs without error, but the signal that made it useful disappears. That makes it one of the easiest patterns to misconfigure silently in production, and one of the cleanest interview probes for whether a candidate has actually used it versus just read about it.
The question targets the relationship between decoding parameters and the assumptions a higher-level prompting pattern makes about the sampler. The right answer is not that T = 0 is bad in general, it is that T = 0 violates the implicit contract self-consistency makes with the decoder. Once that contract is named, the failure mode becomes obvious.
The deep dive walks through the mechanism, the math of why greedy collapses voting, the production bug pattern, and how to wire the contract into your prompt config so the misconfiguration cannot happen silently.
The mechanism, sampling vs greedy
Language models predict a distribution over next tokens conditioned on the prefix. The decoder turns that distribution into a realized token. Sampling decoders draw from the distribution; greedy decoders take the argmax. The choice of decoder is controlled at the API layer by the temperature parameter.
The softmax with temperature is the standard transform:
At T = 1 you sample from the model's raw distribution. T below 1 sharpens the distribution toward the top tokens. T above 1 flattens it. The boundary case is T approaching 0, which concentrates all probability mass on the argmax token. Sampling from a one-hot distribution always returns the same token, so the decoder becomes a deterministic function of the prefix.
The practical consequence is that running the same prompt N times at T = 0 produces N identical outputs. Every reasoning step, every word choice, every final answer is the same. The sampling loop runs, but the outcome is fixed by the prefix.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- LangChain's SelfConsistencyChain and DSPy's BootstrapFewShot both expose temperature as a required knob and warn against N > 1 at T = 0.
- Anthropic's Claude Opus 4.7 and OpenAI's GPT-5.5 both expose temperature per request, so a self-consistency wrapper around either model has to pin T explicitly.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is a good rule for picking T in a self-consistency setup?
Start at T = 0.7 with N = 8 and tune both jointly; lower T narrows diversity, higher N raises cost without proportional gain.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Wiring self-consistency on top of an existing T = 0 prompt and never noticing that the vote is across N identical samples, so the pattern is paying N times the cost for one effective sample.
60 second bullets to scan on the way to the call.
Why greedy decoding is deterministic
The role of softmax with temperature in producing diversity
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.