Loss Masking

Why does completion-only loss usually outperform full-sequence loss on the same SFT data?

Spot the bug: chat-template code that drops the EOS from the labels.

Walk through one training example for SFT'ing a single weather-tool call.

Should the system message contribute to the SFT loss?

Define supervised fine-tuning (SFT) in one breath.

`max_seq_len` controls one bound, what happens to examples that exceed it?

Which loss function does supervised fine-tuning actually minimize?

Why does dropping the EOS token from SFT labels produce a model that never stops generating?

DeepSeek-style reasoning distillation: what travels from teacher to student, and what gets cut from the loss?

DataCollator pads to max_length and sets attention_mask = ones: find what breaks

A batch packs a causal LM with right-padded sequences and applies only the causal mask. Spot the mistake.

Spot the error in this description of SFT loss masking

Explain SFT's loss and the role of prompt-token masking

Spot the bug in this sequence-packing setup

Spot the bug: a Llama-3 fine-tune that skips the end-of-turn token