Why does self-consistency require temperature > 0 and what specifically happens if you set T = 0?

**Self-consistency** is a prompting pattern that samples N different reasoning paths from the model, extracts the final answer from each, and picks whichever answer appears most often. The intuition is that wrong reasoning chains tend to disagree with each other while correct chains tend to converge on the same answer, so voting wins. The pattern only works if the N samples are actually different. That difference comes from sampling stochasticity, which only exists when temperature is above zero. At T > 0 the model samples its next token from a probability distribution, so the same prompt can produce different chains across runs. At T = 0 the model is greedy, meaning it always picks the highest-probability next token. The decoding becomes deterministic. Given the same prompt, all N samples come out identical character for character. Voting across N copies of the same thing is the same as taking one sample. The pattern still runs, but it gives you no benefit while costing N times more.

Self-consistency assumes the N reasoning paths are actually diverse, so the vote across their final answers carries information. That assumption depends on the decoder being stochastic. At T > 0 the softmax produces a real distribution and sampling picks from it, so two runs of the same prompt explore different token sequences. Different sequences lead to different intermediate reasoning, which lead to different final answers, which is what voting needs. Production setups commonly use T = 0.7 with N between 5 and 40. At T = 0 decoding degenerates to greedy argmax. Given the same prompt, the model always picks the same token, then the same next token, all the way down. All N samples are bit for bit identical. The vote across N copies of one sample is mathematically equivalent to taking one sample. You pay N times the API cost and get no diversity benefit, no error correction, no robustness gain. The pattern still runs without error, which is the failure mode people miss. Logs look fine, the vote returns an answer, eval looks stable. The signal is just gone. The correct fix is to assert T > 0 whenever the sampling N exceeds 1.