Match EAGLE-3 and Medusa-2 to drafting mechanism, integration cost and acceptance behavior

Match pairs·Medium·4.0 · 0·~2 min·Asked atDroom Midjourney Zoho·Relevant atNVIDIA

Attempt it

Drag each answer to line up with its matching prompt

EAGLE-3 drafting site

Medusa-2 is simpler to add (heads share the trunk); EAGLE-3 needs hidden-state plumbing plus the small autoregressive draft head

Medusa-2 drafting site

Both verify a token tree in a single target forward pass, but EAGLE-3's tree is typically deeper because acceptance is higher

Verification shape

EAGLE-3 leads on acceptance rate and decode speedup; Medusa-2 trails on acceptance but is easier to ship and tune

Integration cost

Multiple parallel decoding heads attached to the target model, each predicting one of the next several tokens

Reported headline win

Small autoregressive draft head that reads fused hidden states from low, middle and high target layers and predicts tokens directly; no separate draft model required

Classical speculative decoding uses a small draft model to propose tokens that the large target model then verifies. EAGLE-3 and Medusa-2 skip the separate draft model and attach drafting machinery to the target itself. **Medusa-2** adds several extra decoding heads to the target model. Each head predicts one later position in parallel: head 1 predicts t+2, head 2 predicts t+3, and so on, while the normal output head predicts t+1. The target then verifies all those guesses in one forward pass. Serving is simple, but Medusa-2 fine-tunes the base model together with the heads. **EAGLE-3** uses a small autoregressive draft head. It reads hidden states from several of the target's layers and predicts the next few tokens one after another. Because the drafter sees richer information and its own earlier drafts, its guesses match the target more often, so the target accepts more tokens per verification step. EAGLE-3 reports bigger speedups (up to 6.5x) than Medusa-2 (2.3x to 2.8x); Medusa-2 is easier to add and tune. Both verify token trees (multiple candidate continuations) in a single target forward pass to maximize accepted tokens per pass.

EAGLE-3 and Medusa-2 are self-drafting speculative decoding methods: instead of a separate small model proposing tokens, both attach drafting machinery to the target model. They differ in what the drafter sees and how the verification tree is built. **Medusa-2** adds several lightweight decoding heads to the target. Head k is a small network that, from the last hidden state at position t, predicts the token at position t+k+1, while the original LM head predicts t+1. The heads' top candidates form a fixed tree that the target verifies in one forward pass with tree attention. Medusa-1 trains the heads on a frozen backbone; Medusa-2 trains heads and backbone together. **EAGLE-3** uses one small autoregressive draft head. It reads hidden states from the target's low, middle and high layers (multi-layer feature fusion) and predicts tokens directly, rather than predicting the next hidden state as EAGLE-1 did. Training-time test lets the head learn from its own earlier drafts. It grows its tree dynamically from draft confidence, as EAGLE-2 does, and leaves the target weights unchanged. Verification is the same idea in both: the target processes the token tree in one forward pass and keeps the longest branch it accepts. More accepted tokens per pass means a larger wall-clock speedup. EAGLE-3 reports up to 6.5x, about 1.4x over EAGLE-2; the Medusa paper reports 2.3x to 2.8x for Medusa-2.

TL;DR

EAGLE-3 drafts tokens with a small autoregressive head that reads hidden states from several target layers; Medusa-2 drafts with parallel heads on the last hidden state, trained jointly with the model.

Memory aid

Easy to grasp

Two ways to guess what a careful writer is about to say next. Medusa-2 attaches a small panel of helpers to the writer's elbow, each calling out a different upcoming word. The writer then checks the whole guessed phrase at once. The helpers are easy to add and speed things up, but the writer is retrained a little so they fit in, and they sometimes guess the wrong words. EAGLE-3 listens to the writer's thoughts at several stages (hidden states from several layers) and uses a tiny apprentice writer to draft the next few words in sequence. Because the apprentice has richer information, more of its drafts match the writer, so the writer accepts longer chunks at once. EAGLE-3 takes more wiring but accepts more, so each check yields more words.

Key concepts

Speculative decoding EAGLE / EAGLE-3 Medusa / Medusa-2 Draft model

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

EAGLE-3 and Medusa-2 are self-drafting speculative decoding methods. The classical speculative-decoding pattern (Leviathan et al. 2023) used a separate small model as the drafter, for example a 1B model proposing tokens that a 70B target then verified. Self-drafting methods get rid of that separate model and attach drafting machinery directly to the target itself, which simplifies deployment and changes the design tradeoffs.

The central question the two methods answer differently: where should the drafter live and what should it see? Medusa-2 puts the drafter as close to the trunk as possible (extra heads on the target) and feeds it the minimum context (the last hidden state). EAGLE-3 puts the drafter slightly farther out (a small autoregressive head) but feeds it richer context: fused hidden states from several target layers, plus its own earlier drafts.

Self-drafting versus classical speculative decoding

Classical speculative decoding (Leviathan et al. 2023, Chen et al. 2023) runs a small separate model alongside the target. The draft model proposes k tokens autoregressively, the target verifies them in one forward pass, and accepted tokens are kept. If the draft and target distributions match well enough, you get many tokens per target forward pass and the wall-clock decode time drops.

The operational problem with classical speculative decoding is the separate draft model. You have to maintain, version, and align it with the target. You also have to keep it small enough that its inference overhead does not cancel the speedup, while large enough to track the target's distribution. This is a real engineering cost for a serving stack.

Self-drafting methods (Medusa, EAGLE, and others) remove the separate draft model. The drafter is built into or attached to the target. Medusa attaches parallel decoding heads to the target's trunk; EAGLE attaches a small autoregressive head that consumes the target's hidden states. In both cases there is one main model to deploy. The drafter is trained against the target itself, which helps it match the target's distribution.

The shared structure: the drafter proposes a tree of candidate continuations, the target verifies the tree in a single forward pass with an attention mask that lets it process all branches in parallel, and the longest accepted branch is kept. Medusa-2 and EAGLE-3 differ in how they build that tree.

Medusa-2: parallel heads on the trunk

EAGLE-3: autoregressive drafting over hidden states

Tree-based verification: why both methods need it

The production tradeoff

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Property	Medusa-2	EAGLE-3
Drafting mechanism	Parallel lightweight heads on the trunk	Small autoregressive head over hidden states
Draft input	Last target hidden state (single point)	Fused low, middle and high-layer hidden states plus earlier draft tokens
Drafted output shape	Tree of token guesses (shallow)	Tree of token guesses (deeper, dynamic)
Acceptance rate	Moderate, falls off fast with depth	Higher, stays high at greater depth
Integration cost	Low: train heads, add scheduler	Higher: hidden-state plumbing plus AR draft head
Reported speedup	2.3-2.8x (Medusa-2, ICML 2024)	Up to 6.5x, about 1.4x over EAGLE-2

Real products, models, and research that use this idea.

vLLM accepts eagle3 as a speculative decoding method; its current speculative decoding docs list EAGLE, MTP, n-gram and draft-model methods but not Medusa.
The EAGLE-3 paper reports a 1.38x throughput gain at batch size 64 in SGLang, so the gain survives at moderate batch sizes.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does drafting from hidden states raise acceptance over drafting from a single token?

Hidden states encode the full distributional information the target uses to sample its next token, including signal about alternative high-probability tokens. A drafter with access to that richer state can produce continuations that align with the target's actual distribution rather than guessing from a single sampled token, so its proposals match the target more often.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Calling Medusa and EAGLE 'small draft models'. Neither is a separate draft model in the classical sense; both attach drafting machinery to the target model itself. Medusa adds extra heads; EAGLE adds a small autoregressive head that reads the target's hidden states.

60 second bullets to scan on the way to the call.

What 'self-drafting' means (no separate draft model)
Where Medusa-2 puts its drafter (parallel heads on the trunk)

Primary sources. Browse if you want the original framing.