Spot the error in this description of SFT loss masking

Spot the error·Hard·4.0 · 0·~2 min·Relevant atDatabricks Meta OpenAI

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

Quick reference

TL;DR

SFT is next-token cross-entropy over the full sequence. Masking sets prompt-token labels to -100, so only response tokens drive the gradient, not the reverse.

Memory aid

Easy to grasp

Imagine grading a student who copies the exam question, then writes an answer underneath. You only want to score the answer they wrote, not the question they copied. So you cross out the question with a special mark that says 'ignore this' and grade only the answer. SFT works the same way. The model reads the whole page, prompt plus answer, but you mask the prompt tokens with a -100 label so they earn no penalty. Only the answer tokens shape the model's habits. The buggy paragraph flips this: it claims you mask the answer and grade the question, which would train the model to predict prompts and learn nothing useful about responding.

Key concepts

Supervised fine-tuning Loss masking Cross-entropy loss Preference optimization

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 min: identify the contrastive-loss error, the full-sequence error, then the paired mask inversion; explain why prompt labels go to -100 and the left-shift label mechanics.

Code example

python

# labels: prompt tokens masked to -100, response tokens kept
labels = input_ids.clone()
labels[:prompt_len] = -100          # ignore prompt in the loss
# next-token shift handled inside the model: logits[t] predicts labels[t+1]
loss = F.cross_entropy(
    logits[..., :-1, :].reshape(-1, vocab),
    labels[..., 1:].reshape(-1),
    ignore_index=-100,
)

Real-world examples

Real products, models, and research that use this idea.

Hugging Face TRL's SFTTrainer sets prompt-token labels to -100 via DataCollatorForCompletionOnlyLM, so only completion tokens contribute to the loss.
Axolotl and Llama Factory both expose a train_on_inputs flag; leaving it false masks the instruction tokens and trains only on the assistant turn.
Meta's Llama 4 instruction-tuning recipes mask everything except assistant turns in multi-turn chat templates before computing cross-entropy.
PyTorch's nn.CrossEntropyLoss uses ignore_index=-100 by default, which is why -100 is the universal SFT label-mask sentinel.
DeepSeek V4 and Qwen post-training stacks apply per-turn loss masks so user and system tokens never enter the SFT gradient.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy is -100 the magic number for the ignore label, and what breaks if you use 0 instead?

QHow does the input-to-label left shift interact with the prompt mask at the boundary token?

QWhen would training on prompt tokens actually help rather than hurt?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Confusing SFT with preference learning, or inverting the mask so prompt tokens drive the gradient. SFT is next-token cross-entropy with prompt labels set to ignore.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.