Explain SFT's loss and the role of prompt-token masking

**SFT** trains a model on example pairs of an instruction and a good response. The loss is the same one used in pretraining: next-token cross-entropy. For every position in the text, the model guesses the next token. Cross-entropy measures how wrong that guess was. You add up the penalty across positions and push the weights to make the right token more likely. Here is the twist. Your training text is the prompt glued to the response. You do not actually want to teach the model to generate the prompt, the user already types that. So you mask the prompt tokens. Masking means setting the target to a special value, -100 in PyTorch, that the loss function skips. Prompt positions contribute nothing to the loss. Only the response tokens shape the gradient. The payoff: every bit of learning effort goes into producing good responses, not into reciting prompts the model will always be handed at inference. On small datasets this converges faster and reaches slightly better quality.

**SFT** computes ordinary next-token cross-entropy over a tokenised prompt-plus-response sequence. It is the exact pretraining objective, not a new loss. At each position the model outputs a distribution over the vocabulary for the next token. Cross-entropy scores that distribution against the true next token. The penalty is summed over positions, then reduced to a scalar the optimiser minimises. Completion-only loss, or prompt masking, sets the target label to -100 on every prompt token. PyTorch treats -100 as ignore_index, so those positions are dropped before reduction. Only response tokens contribute gradient. The failure mode it prevents is wasted signal. Without masking, the model is rewarded for predicting prompt tokens it will always receive verbatim at inference. That gradient teaches nothing useful and dilutes the signal on the response tokens, which is the behaviour you want to shape. The effect is largest on small datasets. There, masking converges faster and reaches slightly higher quality. On very large datasets the gap shrinks because response tokens dominate anyway. Masking stays the default, since enabling it is essentially free.