Drag each answer to line up with its matching prompt
Pretraining objective
A curated set of prompt-response pairs, orders of magnitude smaller than the pretraining corpus.
SFT objective
Next-token cross-entropy on every token of raw text.
Pretraining data
Next-token cross-entropy on the chosen response tokens only, with the prompt tokens masked out.
SFT data
Trillions of unlabeled tokens from web pages, books, and code.
Pretraining teaches broad next-token prediction from unlabeled corpora, while SFT teaches instruction-following behavior from labeled prompt-answer data.
Imagine teaching a chef in two phases. First, they taste thousands of dishes and learn general cooking patterns without anyone grading their work. That is pretraining. Later, a head chef gives specific customer requests and an example of the right dish for each, then scores how well the chef follows instructions. That is SFT, short for supervised fine-tuning. The first phase builds broad food knowledge, and the second phase shapes how the chef responds to requests politely and usefully. A chef who never tasted enough dishes cannot be fixed by a few lessons in manners.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Pretraining and supervised fine-tuning (SFT) both train the model with next-token cross-entropy. So it is easy to think of them as one process at two sizes. They are different stages. The loss formula is the same, but the source of the target tokens, the tokens that count toward the loss, and the behavior that changes are all different.
Pretraining learns to continue raw text from a very large unlabeled corpus. That is where the model gets its language skill and most of its knowledge. SFT learns to answer prompts from a small, curated set of prompt-response pairs. That is where the model gets its assistant behavior: following instructions, using a chat format, keeping a tone, and stopping at the end of a turn. When you know which stage controls which behavior, you know where to look when a model fails.
One loss formula, two sources of targets
In pretraining, each training sequence is a slice of a document: a web page, a book, a code file. The model predicts every next token, and the loss is the mean negative log-likelihood over all T tokens:
Nobody chose these targets for the model. They are whatever the authors of the documents wrote, for any reason. To lower this loss, the model must learn grammar, facts, code syntax, ways of arguing, and the statistics of many domains. No label says what is correct. The next token of the text is the only signal.
In SFT, each sequence is a prompt followed by a response, and the response was chosen on purpose. A human annotator wrote it, or a filter or a stronger model picked it as the desired answer. The formula is still cross-entropy, but the targets now show one specific behavior: read the request, then answer it in the wanted format and tone. The gradient pushes the model toward whatever the targets show, so the choice of targets is the training signal.
This gives a useful test when you compare two stages. Do not ask only which loss they use. Ask who chose the targets and from which distribution. That question separates pretraining, SFT, and later preference optimization far better than the loss formula does.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Axis | Pretraining | SFT |
|---|---|---|
| Data signal | Unlabeled continuation text | Labeled prompt-answer pairs |
| Primary goal | Broad capability and knowledge | Instruction-following behavior |
| Scale | Massive corpus | Curated, much smaller set |
Real products, models, and research that use this idea.
- Llama 3 8B was pretrained on more than 15T tokens. Its 5-shot MMLU is 66.6 for the base model and 68.4 for the Instruct model, so post-training barely moved knowledge while it changed chat behavior.
- Llama 2 stopped collecting SFT data at 27,540 high-quality annotations, against 2T pretraining tokens.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy can two models with similar pretraining loss still differ after SFT?
Focus on instruction-data quality, rubric consistency, and coverage of user intents rather than base loss alone.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
SFT mostly reshapes how a model answers. It does not teach broad new world knowledge, which comes from pretraining at a scale that SFT data never reaches.
60 second bullets to scan on the way to the call.
Objective difference: pretraining vs SFT
Dataset difference: unlabeled vs labeled
Primary sources. Browse if you want the original framing.
- Ouyang et al., Training language models to follow instructions with human feedback
- Touvron et al., Llama 2: Open Foundation and Fine-Tuned Chat Models
- Zhou et al., LIMA: Less Is More for Alignment
- Llama Team, AI @ Meta, The Llama 3 Herd of Models
- Gekhman et al., Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
Same topic, related formats. Practice these next.