Click any words you think contain an error. Click again to unmark.
SFT changes the data (curated prompt-response pairs), the objective (loss on response tokens given the prompt) and the dynamics (small set, few epochs, low learning rate).
Pretraining is like a student who reads a whole library and guesses the next word on every page. The student picks up facts, grammar and style from everything. SFT is like that student then practicing with a tutor on a short stack of question-and-answer cards. The tutor grades only the student's answers, not the questions read aloud. The stack is small, so the student goes through it a couple of times, slowly and carefully. Cramming the same cards too hard would make the student forget the library and parrot the cards. It is the same student with the same habit of guessing the next word. The material, the part that gets graded, and the pace are all different.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Pretraining and supervised fine-tuning (SFT) share a model, an architecture and a loss formula, so it is easy to describe SFT as more of the same. The shared formula hides three real differences: the data the model sees, which tokens the loss is computed on, and how the optimization behaves.
Each difference has practical effects. Together they decide what SFT can teach a model, what it cannot, and how an SFT run fails when it goes wrong.
One loss formula, two stages
Pretraining trains a decoder-only model to predict every next token in a long stream of raw text:
Every token in the stream is both context and target. Web pages, books, code and papers all contribute, and the model learns grammar, facts, reasoning patterns and style from all of them at once.
SFT starts from the pretrained weights and keeps the same cross-entropy machinery. The forward pass, the softmax over the vocabulary and the backward pass do not change. This is the part of the claim that sounds right, and the reason SFT is sometimes called continued training.
The similarity stops there. What goes into the batch, which positions are scored, and how hard and how long the optimizer pushes are all different.
A pretraining sequence might be 4,096 tokens cut from the end of a web page, a book chapter and the start of another document, joined by end-of-text tokens. The model predicts all 4,096 targets. An SFT sequence is one conversation: a system message, a user question of perhaps 40 tokens, and an assistant answer of perhaps 160 tokens, each marked by template tokens. Only the 160 answer tokens are targets.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Llama 2-Chat: SFT on 27,540 annotated examples for 2 epochs at a learning rate of 2e-5, with the loss zeroed on prompt tokens, after 2T tokens of pretraining.
- LIMA: a 65B LLaMA fine-tuned on 1,000 curated prompt-response pairs reached strong instruction following, supporting the view that knowledge comes from pretraining.
What an interviewer would ask next. Try answering before peeking at the approach.
QShould the SFT loss include the prompt tokens?
Masking makes the objective conditional on the prompt; some recipes keep a small prompt loss for tiny datasets. Compare both on held-out instructions.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Calling SFT more pretraining. The loss is still next-token cross-entropy, but on curated prompt-response pairs, usually scored only on the response, with far fewer steps and a much lower learning rate.
60 second bullets to scan on the way to the call.
Pretraining data versus SFT data
Loss masking on prompt tokens
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.