Spot the errors in this plan to distill reasoning into a 7B student

Spot the error·Hard·4.0 · 0·~2 min·Asked atComet Ml Zed

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

The plan contains three confident assertions, and each one is the opposite of what the DeepSeek-R1 team reported when they ran the actual experiment. First, supervised imitation on reasoning traces is not memorization. The traces are long, varied, and include backtracking and self-correction. SFT on them teaches the student the *behavior*, not specific answers. The 800,000-sample trace dataset used in the R1 distillation includes problems the student will never see at eval time. Second, small models do not discover long-chain reasoning under RL with verifiable rewards. The signal RL needs, occasional correct long chains by accident, almost never appears at 7B scale on hard problems. The R1 team tested this directly with Qwen2.5-Math-7B and reported that distillation beat direct RL at the same compute. Third, imitation learning is bounded above by the teacher distribution. The student fits the teacher's traces; it does not invent better traces. Surpassing the teacher requires either a new RL run with verifier signal beyond what the traces encode or new high-quality data the teacher did not produce. Expecting surpass by default is wrong.

**Each clause flips a published finding from the R1 distillation work.** Clause one calls SFT on teacher traces 'just memorization' and proposes skipping it. The R1 release made the opposite case empirically: the distilled SFT-only checkpoints (Qwen-7B, Llama-8B, Qwen-32B) reached AIME and MATH numbers that beat much larger non-reasoning models. The behavior being installed is the long-deliberation policy: when to verify, when to backtrack, when to re-plan. That is a policy transfer, not a lookup-table memorization. Clause two claims direct RL on the 7B is just as good. The R1 paper explicitly ran that ablation. Applying their RL recipe to Qwen2.5-Math-7B underperformed the trace-distilled equivalent at matched compute. The reason is structural: the success-trace rate at small scale is too low for verifier reward to climb. RL needs a base strong enough that long sampled chains occasionally hit the right answer. Clause three predicts the student surpasses its teacher on AIME. Imitation learning is bounded above by the teacher's policy in distribution. The student can approach the teacher and occasionally beat it on a noisy benchmark slice, but systematic surpassing requires adding signal: new data, new RL with rewards beyond the trace distribution, or test-time scaling. Without those, expecting the student to exceed the teacher is wishful. The corrected plan: SFT on curated teacher traces first, then optional GRPO refinement on top, with explicit acknowledgement that the teacher is the imitation ceiling.

TL;DR

All three claims invert published findings: trace SFT is the proven recipe, direct RL on small models underperforms distillation, and the student is bounded near the teacher unless extended with new data or RL.

Memory aid

Easy to grasp

Think of cooking school. Claim one says watching a master chef and copying is just memorizing recipes. In reality, that is how you learn the craft. Claim two says a beginner with a stove and a cookbook will figure out new dishes on their own as well as the master did. In reality, the beginner needs the demonstrations first. Claim three says the student will end up a better chef than their teacher just by copying. In reality, copying gets you close to the teacher but not past them. Each claim sounds confident, but each one inverts what actually happened when researchers tried it on real reasoning models.

Key concepts

Reasoning Models

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Spot-the-error questions on reasoning distillation are common in 2026 senior screens because the topic has accumulated a layer of confident folk wisdom that is the opposite of what the published experiments showed. The plan in this question is a compressed version of that folk wisdom: three confident clauses, each one cleanly contradicted by the DeepSeek-R1 ablation table.

This explanation walks each error through its specific contradiction, the mechanism behind why the intuition fails, and what the corrected plan looks like in practice. The aim is not just to refute the three claims but to leave you with the right mental model for why distillation works and where its ceiling actually sits.

Clause one: SFT on traces is not memorization

The plan dismisses trace SFT as 'supervised imitation only memorizes.' This frames imitation learning as a lookup table, which mischaracterizes both the data and the training objective.

Reasoning traces are long, conditional on the input problem, and structurally varied. A single problem can yield ten different valid traces with different branching, backtracking, and verification patterns. Memorization would require the model to store the mapping from input to output verbatim, but at the trace lengths involved (often 2,000 to 20,000 tokens), and at the dataset scales involved (hundreds of thousands of traces), the model does not have the parameter count for verbatim storage of more than a small fraction of the data. What it stores is the conditional pattern.

The held-out evaluation backs this up. The R1 distilled 7B reaches strong AIME 2024 numbers, but AIME 2024 problems were not in the teacher's training set or the distillation trace set in any verbatim form. If the model were memorizing, those held-out scores would be near zero. They are not; they beat much larger non-reasoning models.

The correct mental model is policy transfer. The teacher discovered a reasoning policy through expensive RL. The traces are samples from that policy. SFT compresses the policy into the student's weights by next-token prediction. The student then applies the policy to new inputs, including held-out benchmarks.

Diagnostic check: if the plan's claim were true, distilled-student AIME numbers should crash on held-out years (AIME 2025 or 2026 problems released after the trace collection). They do not; they degrade only modestly, which is exactly the signature of policy transfer rather than memorization.

Clause two: small models do not discover reasoning under direct RL

Clause three: imitation is bounded by the teacher

The corrected plan and how to defend it

Common false claims about reasoning distillation

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

DeepSeek-R1-Distill-Qwen-7B and -Llama-8B (January 2025) ship as SFT-only distilled reasoning models; their numbers are the empirical refutation of the plan.
The s1 paper (Muennighoff et al., 2025) demonstrated that surpassing-style gains require test-time scaling on top of distillation, not pure SFT.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you detect the moment in training where GRPO advantage signal collapses to noise on a small base?

Track per-group reward variance and the fraction of groups containing at least one correct rollout; once that fraction drops below a threshold (often 10 to 20 percent), the gradient is mostly entropy-driven and you should fall back to SFT or curriculum simplification.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Reading the plan as internally consistent because each clause sounds intuitive in isolation; the published ablation contradicts each one specifically.

60 second bullets to scan on the way to the call.

Why SFT on reasoning traces is policy transfer, not memorization
What the R1 paper's direct-RL-on-7B ablation showed

Primary sources. Browse if you want the original framing.