Loss Masking

PremiumWhy does completion only…

PremiumSpot the bug: chat…

Walk through one training example for SFT'ing a single weather tool call.

Should the system message contribute to the SFT loss?

Define supervised fine-tuning (SFT) in one breath.

`max_seq_len` controls one bound, what happens to examples that exceed it?

Which loss function does supervised fine-tuning actually minimize?

Why does dropping the EOS token from SFT labels produce a model that never stops generating?

DeepSeek style reasoning distillation: what travels from teacher to student, and what gets cut from the loss?

DataCollator pads to max_length and sets attention_mask = ones: find what breaks

A batch packs a causal LM with right padded sequences and applies only the causal mask. Spot the mistake.

Spot the error in this description of SFT loss masking

Explain SFT's loss and the role of prompt token masking

Spot the bug in this sequence packing setup

Spot the bug: a Llama-3 fine-tune that skips the end of turn token