**SFT** trains a language model on demonstrations: a prompt and an ideal response. The loss is the same one used during pretraining, applied to the response.
At each position in the response, the model produces a probability distribution over its vocabulary, often 50,000 to 200,000 tokens, and predicts the next token. The loss for that position is the negative log probability the model assigned to the true next token. Big number means the model was very surprised; small number means it was confident in the right answer.
The total loss for one example is that quantity summed over every response position, then averaged across the batch. Prompt and system tokens are masked out, so they do not contribute to the loss. The model still sees them as context but is not graded on its predictions there.
This is **token level cross-entropy**, the standard categorical loss for predicting from a discrete vocabulary. It is the right loss because the target is one specific token at each position, not a continuous value, so a categorical loss makes sense and a regression loss like MSE does not.
**Supervised fine-tuning** minimizes token level cross-entropy on the response tokens. Nothing more exotic.
For a single example with response tokens `y_1, ..., y_T` and prefix `x`, the loss is:
$$\mathcal{L}_{\text{SFT}} = -\sum_{t=1}^{T} m_t \cdot \log p_\theta\bigl(y_t \mid x, y_{<t}\bigr)$$
The mask `m_t` is 1 for response positions and 0 for prompt and system positions, so only the response contributes. Teacher forcing means at training time the model sees the ground truth prefix when predicting each token, not its own previous predictions.
This is exactly the pretraining loss, applied to a smaller curated dataset. That alignment is intentional. Fine-tuning has to match the pretraining objective or the embedding space and the loss disagree, and the model unlearns more than it learns.
The other options each describe a different training phase. MSE is for regression, not categorical prediction. The triplet contrastive loss is DPO, which needs paired chosen and rejected responses. The KL divergence to a frozen reference is the PPO regularizer in RLHF, also pair or reward based. SFT is the simpler step that often comes before either of those.
Quick reference
TL;DR
Supervised fine-tuning minimizes token-level cross-entropy on the response tokens, the same objective the base model was pretrained with, just on a smaller curated set.
Imagine teaching someone to copy a famous handwriting sample one letter at a time. At each letter you cover the answer, ask them to guess the next stroke, then reveal the correct one. The lesson scores how surprised they were by the actual letter: very surprised is a big penalty, perfectly expected is a small one. They keep practicing until their guesses match the handwriting closely. SFT works the same way with words. The model reads a prompt, then one word at a time tries to predict the next word in the ideal response. The loss is the surprise score, summed over every word in the response. The system and user prompts are not scored, only the answer is, because the answer is what you are teaching the model to write.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Aligning a model to human preferences from paired chosen and rejected responses SFT cannot encode a preference signal; DPO or ORPO or PPO RLHF is the appropriate objective for preference data.
Distilling a black-box teacher with only ranking feedback rather than full responses The SFT loss requires a concrete target sequence; ranking signals need a preference style objective instead.
Constrained generation where the model must satisfy hard format rules that the loss cannot reward directly Cross-entropy on the response cannot express format level constraints, so post-hoc validators or reward shaping are needed.
Training a classifier head on top of an encoder for binary or multi-class prediction That is task-specific cross-entropy on a single classification target, not the autoregressive next-token loss SFT uses.
Quick scan · 2–4 min
Everything important, quickly.
4 min: name token level cross-entropy with masking, write the formula, explain teacher forcing, contrast with DPO and PPO losses, and call out the chat-template masking bug as the common production failure mode.
Real-world examples
Real products, models, and research that use this idea.
1Hugging Face TRL SFTTrainer wraps PyTorch CrossEntropyLoss with an ignore-index of -100 for prompt positions, the standard recipe used to fine-tune Llama 4 and Mistral on instruction data.
2Axolotl exposes a train_on_inputs flag that toggles whether prompt tokens contribute to the loss, defaulting to false so only response cross-entropy is minimized.
3DeepSeek V4 distillation recipes report training loss as mean next-token cross-entropy averaged across the response positions of the curated student dataset.
4Unsloth tutorials walk users through inspecting the loss mask alignment to confirm the chat template did not silently grade the system prompt during a Gemma 4 SFT run.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is it important that the SFT loss exactly matches the pretraining loss objective?
Sign in or go Pro to see the approach for this follow-up.
QHow does loss masking on prompt tokens affect convergence and what changes if you grade prompts too?
Sign in or go Pro to see the approach for this follow-up.
QWhat goes wrong if you set the ignore-index incorrectly when packing multiple examples into one sequence?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Conflating the SFT loss with the DPO pairwise loss or the PPO KL term. SFT is plain next-token cross-entropy on a single ideal response, with no pair, no reference, and no reward model.