Use the speedup formula S = (1 - α^(K+1)) / ((1 - α) * (1 + K * c)).
Compute S to two decimal places for these parameters:
- α (acceptance rate) = 0.7
- K (draft length) = 4
- c (target/draft cost ratio, i.e. draft cost as a fraction of target cost) = 0.1
Report S as a number with two decimals. Then state whether speculative decoding wins (S > 1) or loses (S < 1) at these settings.
**Speculative decoding** uses a small, fast draft model to propose several tokens, then the big target model verifies all of them in a single forward pass. Accepted tokens are kept; the first rejected one is corrected and the rest are discarded.
The win comes from verifying many tokens per expensive target pass instead of one. The acceptance rate, written α, is how often a drafted token survives verification. A higher α means more tokens accepted per round, so more speedup.
The formula here balances two things. The numerator counts expected tokens confirmed per verify round. The denominator charges the cost of running the draft model K times, scaled by c, the draft-to-target cost ratio.
Plugging in α=0.7, K=4, c=0.1: the numerator is 1 − 0.7^5 = 0.83193, the denominator is 0.3 × 1.4 = 0.42, and S ≈ 1.98. Since S is above 1, speculative decoding nearly doubles throughput at these settings, so it clearly wins.
Compute it piece by piece. The numerator is 1 − α^(K+1) = 1 − 0.7^5 = 1 − 0.16807 = 0.83193. The denominator is (1 − α)(1 + K·c) = 0.3 × 1.4 = 0.42. So S ≈ 1.98, and since that exceeds 1, speculative decoding wins here.
The intuition behind the formula matters more than the number. The expected tokens accepted before the first rejection follows a geometric pattern in α. Including the one guaranteed token from the target, expected tokens per verify is:
$$E = \frac{1 - \alpha^{K+1}}{1 - \alpha}$$
That is the numerator. The denominator divides by 1 to charge one target pass, plus K·c for running the draft model K times. So S is throughput gain per unit target cost.
The verdict is fragile in α. At α=0.4 you get S ≈ 1.18, only marginal. At α=0.2 the scheme drops below 1 and loses. Draft length K and cost ratio c matter, but acceptance rate dominates, which is why teams measure α empirically before committing to a draft model.
Quick reference
TL;DR
Plug α=0.7, K=4, c=0.1 into the speedup formula and S ≈ 1.98, so speculative decoding nearly doubles throughput here and clearly wins.
Imagine a fast intern who guesses the next four words you are about to write, then you check all four at once instead of writing them yourself. If the intern is usually right, you save tons of time, because one check covers several words. If the intern guesses badly, you keep throwing away the guesses and re-doing the work, so the help is wasted. How often the intern is right is the acceptance rate. The longer the guess and the higher the acceptance rate, the more words you confirm per check. But longer guesses also cost a little extra, since the intern still has to write them. The formula in this question just balances how many words you confirm per check against that small extra cost.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Acceptance rate is low because the draft model poorly mimics the target distribution At low alpha the speedup formula falls below 1, so each verify wastes drafted tokens and net throughput drops.
Already running large batches that saturate GPU compute during decode Speculative decoding trades extra compute for latency; a compute-bound batched server has no spare headroom to spend.
Very short generations where draft setup and verify overhead dominate The amortization benefit needs enough tokens; tiny outputs pay the draft cost without recovering it across many rounds.
Strict bitwise-deterministic output requirements across runs Verification keeps the target distribution, but draft scheduling and acceptance can complicate reproducibility guarantees in serving.
Quick scan · 2–4 min
Everything important, quickly.
4 min: derive the numerator as a geometric token count, explain the denominator overhead, plug in the numbers, then discuss alpha sensitivity and how to raise acceptance.
Real-world examples
Real products, models, and research that use this idea.
1vLLM ships speculative decoding with draft models, n-gram proposers, and EAGLE heads, exposing acceptance-rate metrics so operators can tune draft length per workload.
2Medusa adds trained prediction heads to the target model itself, raising acceptance without a separate draft network on Llama-class models.
3EAGLE and EAGLE-2 perform feature-level autoregression to push acceptance rates above naive draft models, widely benchmarked on Llama 4 and Qwen 3.
4TensorRT-LLM (NVIDIA) implements draft-target and Medusa-style speculative decoding as production paths on H100 and B200 serving stacks.
5DeepSeek V4 uses multi-token prediction during training, which doubles as a high-acceptance speculative draft at inference time.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is the exponent K+1 rather than K in the numerator?
Sign in or go Pro to see the approach for this follow-up.
QGiven alpha and c, what draft length K maximizes the speedup S?
Sign in or go Pro to see the approach for this follow-up.
QHow would you raise the acceptance rate alpha in production?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Raising α to the power K instead of K+1, or forgetting the draft overhead term 1 + K·c in the denominator. Both inflate the predicted speedup.