Spot the error in this description of SFT loss masking · Zenaique

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

Quick reference

TL;DR

SFT is next-token cross-entropy over the full sequence. Masking sets prompt-token labels to -100, so only response tokens drive the gradient, not the reverse.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine grading a student who copies the exam question, then writes an answer underneath. You only want to score the answer they wrote, not the question they copied. So you cross out the question with a special mark that says 'ignore this' and grade only the answer. SFT works the same way. The model reads the whole page, prompt plus answer, but you mask the prompt tokens with a -100 label so they earn no penalty. Only the answer tokens shape the model's habits. The buggy paragraph flips this: it claims you mask the answer and grade the question, which would train the model to predict prompts and learn nothing useful about responding.

Key concepts

Supervised fine-tuning Loss masking Cross-entropy loss Preference optimization

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

5 min: identify the contrastive-loss error, the full-sequence error, then the paired mask inversion; explain why prompt labels go to -100 and the left-shift label mechanics.

Code example

python

# labels: prompt tokens masked to -100, response tokens kept
labels = input_ids.clone()
labels[:prompt_len] = -100          # ignore prompt in the loss
# next-token shift handled inside the model: logits[t] predicts labels[t+1]
loss = F.cross_entropy(
    logits[..., :-1, :].reshape(-1, vocab),
    labels[..., 1:].reshape(-1),
    ignore_index=-100,
)

Real-world examples

Real products, models, and research that use this idea.

Hugging Face TRL's SFTTrainer sets prompt-token labels to -100 via DataCollatorForCompletionOnlyLM, so only completion tokens contribute to the loss.
Axolotl and Llama Factory both expose a train_on_inputs flag; leaving it false masks the instruction tokens and trains only on the assistant turn.
Meta's Llama 4 instruction-tuning recipes mask everything except assistant turns in multi-turn chat templates before computing cross-entropy.
PyTorch's nn.CrossEntropyLoss uses ignore_index=-100 by default, which is why -100 is the universal SFT label-mask sentinel.
DeepSeek V4 and Qwen post-training stacks apply per-turn loss masks so user and system tokens never enter the SFT gradient.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy is -100 the magic number for the ignore label, and what breaks if you use 0 instead?

Sign in or go Pro to see the approach for this follow-up.

QHow does the input-to-label left shift interact with the prompt mask at the boundary token?

Sign in or go Pro to see the approach for this follow-up.

QWhen would training on prompt tokens actually help rather than hurt?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Confusing SFT with preference learning, or inverting the mask so prompt tokens drive the gradient. SFT is next-token cross-entropy with prompt labels set to ignore.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.