Describe what the SFT loss computes for a single (prompt, response) training example. What does 'completion only loss' or 'prompt masking' mean, and what failure mode does it prevent?
SFT is next-token cross-entropy over the prompt-plus-response sequence. Prompt masking sets prompt targets to -100 so only response tokens train the model.
Picture a fill-in-the-blank worksheet where the question is already printed and you must write the answer. SFT grades the model on every blank it could fill. But you do not want credit for re-copying the printed question, you already have it. So a teacher draws a line through the question part and only marks the answer you wrote. Prompt masking is that line: it tells the grader to skip the question tokens and score only the response tokens. The model still reads the full question to understand context, it just earns a grade only on the words it was supposed to generate. That way all of the learning effort goes into shaping good answers, not into memorising prompts.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Supervised fine-tuning sounds like it should have an exotic objective. It does not. The loss that shapes an instruction-following model is the same next-token cross-entropy that pretrained it. The interesting part is not the formula, it is which tokens you let the formula see.
The question probes two things at once. First, can you state the loss precisely, including the often-forgotten one-position shift between predictions and targets? Second, do you understand prompt masking, the small bookkeeping trick that decides whether your gradient teaches behaviour or wastes itself reconstructing inputs the model is always handed?
These are exactly the details that separate someone who has run a fine-tune from someone who has only read about it. The masking is a single boolean flag in most frameworks, yet getting it wrong, or misunderstanding what it does, produces slow convergence, silent off-by-one bugs, and models that train on the wrong spans. This deep dive walks the loss end to end, then the masking, the failure mode it prevents, the reduction subtlety, and the practical pitfalls that bite real runs.
The loss: next-token cross-entropy
SFT trains on a sequence formed by concatenating a prompt with its target response, tokenised into one flat array. The model is autoregressive, so at each position it produces a probability distribution over the vocabulary for the very next token.
Cross-entropy scores each of those distributions against the true next token. For a single position, the penalty is the negative log probability the model assigned to the correct token. Lower probability on the right token means a larger penalty and a stronger gradient pushing that token up.
The canonical form, where the target is a one-hot vector over the vocabulary, is:
Summed across every scored position and reduced to a scalar, this is the entire objective. It is identical to the pretraining loss. Nothing about SFT changes the math; the dataset changes, and as we will see, the set of scored positions changes.
One consequence is worth internalising. Because the loss is per-token and additive, every scored position pulls the weights independently toward its target. There is no holistic 'response quality' term, no notion of the answer being good as a whole. The model simply learns, token by token, to make the observed continuations more probable in context. Any higher-level behaviour you observe, a consistent tone or a reliable format, emerges from many local next-token corrections accumulating across the dataset. That is why data quality and coverage dominate SFT outcomes far more than any clever loss variant.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
import torch.nn.functional as F
# logits: [B, T, V]; labels: [B, T] with -100 on prompt tokens
shift_logits = logits[:, :-1, :].contiguous()
shift_labels = labels[:, 1:].contiguous()
loss = F.cross_entropy(
shift_logits.view(-1, shift_logits.size(-1)),
shift_labels.view(-1),
ignore_index=-100, # prompt tokens skipped
)| Aspect | Full-sequence loss | Completion-only loss |
|---|---|---|
| Tokens scored | Prompt plus response | Response only |
| Prompt label | True next token | -100 (ignore_index) |
| Gradient use | Partly spent on prompts | All on response behaviour |
| Small-dataset quality | Slightly worse, slower | Faster, slightly better |
| Large-dataset gap | Shrinks (response dominates) | Still default, near free |
Real products, models, and research that use this idea.
- Hugging Face TRL's SFTTrainer enables completion-only loss via DataCollatorForCompletionOnlyLM, masking prompt tokens to -100 by default.
- Axolotl exposes train_on_inputs: false to mask instruction tokens, the standard recipe used to fine-tune Llama 4 and Qwen3 chat models.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the choice between token-mean and sample-mean reduction change what SFT optimises?
Token-mean weights gradient by response length, so verbose examples dominate. Sample-mean normalises per example. Discuss how this interacts with sequence packing and length-imbalanced datasets.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Saying SFT uses a special loss. It is plain next-token cross-entropy, the same objective as pretraining. The only change is which tokens contribute.
60 second bullets to scan on the way to the call.
SFT loss is next-token cross-entropy, identical to pretraining
The one-position shift between logits and labels
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.