A reasoning API samples N=5 independent chain-of-thought completions for one math problem. Final extracted answers: 42, 42, 37, 42, 42. The serving stack returns the majority vote answer (no verifier reranking). Which integer is returned?
Four of five sampled completions agree on 42, so pure majority vote returns 42 without any verifier reranking or retraining.
Imagine asking five friends the same math question separately. Four say 42 and one says 37. If you pick the answer most people gave, you choose 42. Best-of-N voting at inference time works the same way: sample several chain-of-thought completions, extract each final answer, and return whichever number appears most often. The model weights never change — it is just counting votes after the model already ran five times.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Best-of-N is one of the oldest test-time scaling tricks in the reasoning-model era, yet interview answers still confuse vote aggregation with verifier ranking. This scenario is deliberately minimal: five integers, one rule (majority vote), one correct output.
Understanding why 42 wins — and when it would not — separates candidates who memorized "sample more" from those who can design inference policies.
The sections below build mechanism first, then production tradeoffs, then how teams measure success in 2026 deployments. Read each heading as a promise — by the end you should explain this topic to a colleague designing a reasoning API or post-training run.
What best-of-N does at inference time
Best-of-N means running the same prompt through the model N times with stochastic decoding (or N parallel workers), then combining the results. No weights change. No backward pass. You are buying accuracy with extra forward passes.
For math and code, the combination step usually operates on extracted final answers — the number after the last equals sign, the return value of a function, the multiple-choice letter. The full chain-of-thought may differ across samples while the extracted answer matches.
In this problem, N=5 and extraction yields 42, 42, 37, 42, 42. The aggregation rule stated in the prompt is pure majority vote, not verifier score. That single sentence determines the answer.
Self-consistency papers popularized vote aggregation before reasoning models branded it as test-time compute. The same math applies whether samples come from GPT-4 with temperature 0.7 or from DeepSeek-R1 with long CoT — count modes on extracted finals.
Historical note: Wang et al. self-consistency predates branded reasoning APIs but established vote on math finals as a strong baseline. Modern reasoning models internalize similar aggregation under proprietary wrappers.
Production checkpoint. Before shipping, walk through a concrete scenario with real numbers: who owns the metric dashboard, what fails first under load, and what you would change after one week of live traffic. Interviewers reward answers that connect mechanism to operability — not only definitions. If you can name one 2026 vendor example and one failure mode for this topic, you are already ahead of candidates who stop at textbook recitation.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Self-consistency decoding on GSM8K samples multiple CoT paths and votes on the final numeric answer
- DeepSeek-R1 eval reports often use pass@k and majority vote variants on AIME-style problems
What an interviewer would ask next. Try answering before peeking at the approach.
QWhen would verifier-ranked best-of-N disagree with majority vote?
When the minority completion has a higher verifier score — e.g., one long correct derivation versus four copies of a wrong shortcut. Tie-break policy matters.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Picking the outlier 37 because it looks like a deliberate correction, or confusing majority vote with verifier-ranked best-of-N.
60 second bullets to scan on the way to the call.
State that N=5 independent completions were sampled
List extracted answers and count frequencies
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.