**Self-consistency** is a prompting pattern that samples N different reasoning paths from the model, extracts the final answer from each, and picks whichever answer appears most often. The intuition is that wrong reasoning chains tend to disagree with each other while correct chains tend to converge on the same answer, so voting wins.
The pattern only works if the N samples are actually different. That difference comes from sampling stochasticity, which only exists when temperature is above zero. At T > 0 the model samples its next token from a probability distribution, so the same prompt can produce different chains across runs.
At T = 0 the model is greedy, meaning it always picks the highest-probability next token. The decoding becomes deterministic. Given the same prompt, all N samples come out identical character for character. Voting across N copies of the same thing is the same as taking one sample. The pattern still runs, but it gives you no benefit while costing N times more.
Self-consistency assumes the N reasoning paths are actually diverse, so the vote across their final answers carries information. That assumption depends on the decoder being stochastic.
At T > 0 the softmax produces a real distribution and sampling picks from it, so two runs of the same prompt explore different token sequences. Different sequences lead to different intermediate reasoning, which lead to different final answers, which is what voting needs. Production setups commonly use T = 0.7 with N between 5 and 40.
At T = 0 decoding degenerates to greedy argmax. Given the same prompt, the model always picks the same token, then the same next token, all the way down. All N samples are bit for bit identical. The vote across N copies of one sample is mathematically equivalent to taking one sample. You pay N times the API cost and get no diversity benefit, no error correction, no robustness gain.
The pattern still runs without error, which is the failure mode people miss. Logs look fine, the vote returns an answer, eval looks stable. The signal is just gone. The correct fix is to assert T > 0 whenever the sampling N exceeds 1.
Quick reference
TL;DR
Self-consistency votes across N distinct reasoning paths. At T = 0 the model is deterministic, all N samples are identical, and the vote collapses to a single sample.
Imagine asking ten people the same hard riddle and going with whichever answer the most people gave. That works because each person reasons differently and you get a real majority. Now imagine ten clones of the same person, all thinking in exactly the same way. They will give the same answer every time. There is no real vote, just one opinion repeated. Self-consistency is the first situation, and it needs the model to think a little differently each run. Temperature equals zero turns it into the second situation, with ten identical clones.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Tasks with a single deterministic path from prompt to answer (lookup, extraction) The chains converge to the same answer even at T > 0, so voting buys no signal and just multiplies cost.
Latency-critical user-facing requests Self-consistency multiplies inference cost and latency by N; a single chain at T = 0 is far cheaper when the task does not benefit from voting.
Reproducible eval pipelines that need bit for bit consistency Sampling-based voting changes outputs across runs by design, which fights the goal of reproducibility.
Tasks where a verifier or judge model is available Best-of-N with a verifier typically outperforms majority voting because it uses a stronger signal than vote count.
Quick scan · 2–4 min
Everything important, quickly.
4 min: softmax with temperature + why greedy is deterministic + how voting collapses + production bug pattern + when N > 1 implies T > 0.
Real-world examples
Real products, models, and research that use this idea.
1LangChain's SelfConsistencyChain and DSPy's BootstrapFewShot both expose temperature as a required knob and warn against N > 1 at T = 0.
2Anthropic's Claude Opus 4.7 and OpenAI's GPT-5.5 both expose temperature per request, so a self-consistency wrapper around either model has to pin T explicitly.
3Promptfoo and Braintrust eval harnesses surface T as part of the prompt config so self-consistency runs in eval match prod sampling behavior.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is a good rule for picking T in a self-consistency setup?
Sign in or go Pro to see the approach for this follow-up.
QWhen does self-consistency stop helping even at T > 0?
Sign in or go Pro to see the approach for this follow-up.
QHow would you detect the silent N samples are identical bug in production?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Wiring self-consistency on top of an existing T = 0 prompt and never noticing that the vote is across N identical samples, so the pattern is paying N times the cost for one effective sample.