How does self-consistency improve over single-pass Chain-of-Thought and what's the most important prerequisite for it to work?

Self-consistency is a clever upgrade over plain Chain-of-Thought. Plain CoT runs the model once and reads off the answer. Self-consistency runs the model N times in parallel, lets each run produce its own reasoning path, and votes on the final answer. The intuition is that correct answers tend to be reached through several distinct reasoning paths, while incorrect answers tend to wander to different wrong answers across runs. When you vote on the majority, the correct answer wins more often than a single random run would have. The prerequisite is temperature greater than zero. At temperature zero the model is deterministic; every run produces exactly the same tokens, so N identical samples collapse into one. The voting has nothing to do. Pick a temperature in the 0.5 to 0.9 range, typically 0.7 for math reasoning tasks. The cost is roughly N times the base inference cost. The benefit is a measurable accuracy lift, often 10 to 20 points on reasoning benchmarks. The tradeoff depends on how much the accuracy is worth on your route.

**Self-consistency** samples N independent Chain-of-Thought reasoning paths at temperature greater than zero, then takes the majority vote over the final answers. The mechanism amplifies the reasoning signal that single-pass CoT leaves on the table. The canonical voting formula is the argmax over answer counts across N samples. $$\text{answer} = \arg\max_y \sum_{i=1}^N \mathbb{1}[y_i = y]$$ Each `y_i` is the final answer from one sample. The indicator counts how often each answer appears. Argmax picks the most common one. The hypothesis: correct answers cluster across diverse reasoning paths while incorrect answers scatter. The vote concentrates the signal. **The prerequisite is temperature > 0.** At T=0 the decoder is deterministic. Running the same prompt N times gives N identical samples; voting reduces to a single sample's outcome and the entire mechanism is wasted compute. Typical temperatures for self-consistency are 0.5 to 0.9, with 0.7 the most common production default for math and multi-step reasoning. Empirical gains are large. The original Wang et al. paper reported +17.9 points on GSM8K with PaLM-540B at N=40. Modern reproductions on GPT-4 class and Claude class models show 10 to 20 point lifts on multi-step reasoning benchmarks. The cost is N times the base inference. In production, N=5-10 captures most of the lift at a fraction of the compute, with diminishing returns past N=10.