CoT distillation and STaR: what made rejection-sampled reasoning FT effective?

**Chain-of-thought distillation** teaches a model to reason by training it on worked solutions, not just answers. A strong teacher solves problems out loud, step by step, and a smaller student model trains on those step-by-step traces plus the final answer. Each training example is a triple: the question, the reasoning steps, and the answer. The student learns to imitate the pattern: think first, then commit. It inherits the teacher's reasoning habits without needing to be as large. STaR, the Self-Taught Reasoner, removes the need for a smarter teacher. The model generates its own reasoning traces for problems where the answer is known. You keep only the traces that reached the correct answer and throw the rest away. Then you fine-tune on the kept set and repeat the loop. The key trick is what you check. You never grade the reasoning itself; you only check the final answer. That is cheap for math, code, or facts with known answers. Wrong reasoning rarely lands the right answer often, so over many rounds the kept traces are mostly good reasoning.

**CoT distillation** is supervised fine-tuning on reasoning triples. A teacher produces step-by-step traces for problems with known answers. Each example becomes a question, a trace, and a final answer. The student trains to predict the trace and answer together, inheriting the teacher's reasoning style. STaR removes the stronger teacher. The model samples its own traces for problems with known answers. You keep only traces that reach the correct answer, then fine-tune and repeat. This is a self-improvement loop bootstrapping reasoning from the model's own lucky-correct outputs. The trick that makes the loop work is outcome-based rejection sampling. Judging trace quality needs a separate, biased, hard-to-build evaluator. Checking the final answer needs only ground truth, which is cheap and unambiguous for math, code, and many factual tasks. The catch is false positives: a right answer from flawed reasoning. Across many iterations this noise averages out, because wrong reasoning rarely lands the right answer consistently. STaR also added a backward rationalisation step for failed problems, generating a trace given the answer, which is more controversial. Modern descendants include rejection fine-tuning and DeepSeek-R1-style outcome reinforcement learning with verifier rewards.