Predict whether speculative decoding wins for given α, K, c

**Speculative decoding** uses a small, fast draft model to propose several tokens, then the big target model verifies all of them in a single forward pass. Accepted tokens are kept; the first rejected one is corrected and the rest are discarded. The win comes from verifying many tokens per expensive target pass instead of one. The acceptance rate, written α, is how often a drafted token survives verification. A higher α means more tokens accepted per round, so more speedup. The formula here balances two things. The numerator counts expected tokens confirmed per verify round. The denominator charges the cost of running the draft model K times, scaled by c, the draft-to-target cost ratio. Plugging in α=0.7, K=4, c=0.1: the numerator is 1 − 0.7^5 = 0.83193, the denominator is 0.3 × 1.4 = 0.42, and S ≈ 1.98. Since S is above 1, speculative decoding nearly doubles throughput at these settings, so it clearly wins.

Compute it piece by piece. The numerator is 1 − α^(K+1) = 1 − 0.7^5 = 1 − 0.16807 = 0.83193. The denominator is (1 − α)(1 + K·c) = 0.3 × 1.4 = 0.42. So S ≈ 1.98, and since that exceeds 1, speculative decoding wins here. The intuition behind the formula matters more than the number. The expected tokens accepted before the first rejection follows a geometric pattern in α. Including the one guaranteed token from the target, expected tokens per verify is: $$E = \frac{1 - \alpha^{K+1}}{1 - \alpha}$$ That is the numerator. The denominator divides by 1 to charge one target pass, plus K·c for running the draft model K times. So S is throughput gain per unit target cost. The verdict is fragile in α. At α=0.4 you get S ≈ 1.18, only marginal. At α=0.2 the scheme drops below 1 and loses. Draft length K and cost ratio c matter, but acceptance rate dominates, which is why teams measure α empirically before committing to a draft model.