Why is chat-template integrity the most common 'silent quality killer' in FT?

Modern chat models do not just see plain text. They see **special tokens** that mark where each turn starts, who is speaking, and where a turn ends. ChatML uses `<|im_start|>` and `<|im_end|>`; Llama-3 uses header tokens plus `<|eot_id|>`. These tokens are not decoration. The base model was trained on millions of examples wrapped in exactly this format. It learned strong habits around them: stop generating when you see the end-of-turn token, switch tone when you see the assistant header. When your fine-tuning data uses a slightly different format, you quietly break those habits. Drop the end-of-turn token and the model never learns to stop, so it rambles into a fake next turn. Swap the role header for plain text like `### Assistant:` and the model loses a prior it already had. The nasty part is that nothing looks wrong during training. The loss curve and the gradients look totally normal. You only notice at evaluation, when the model behaves strangely for no obvious reason. The fix is simple discipline. Always format data with the tokenizer's official `apply_chat_template()` helper, and never hand-write the prompt string.

Chat models delimit turns with **special tokens**, like `<|im_end|>` in ChatML or `<|eot_id|>` in Llama-3. These are not cosmetic. The base model trained on this exact format, so the tokens carry learned priors for stopping, role identity, and where to attend. A tiny mismatch breaks three things at once. First, a missing end-of-turn token means the model never learns to stop, so generations bleed into invented turns. Second, a wrong role header discards the pretraining prior on role boundaries, forcing SFT to relearn it with far less data. Third, writing a special token as literal text causes tokenisation skew. The skew is the subtle one. A special token is a single vocab ID. Written as plain text, the tokeniser splits it into several BPE pieces. So the representation seen in training differs from the single injected ID seen at inference. This bug class is silent. Loss and gradient norms look completely normal because the model is learning something coherent, just the wrong thing. You only catch it at eval. The mitigation is discipline. Always use the model's official tokenizer and `apply_chat_template()`. Never build the prompt by hand. Then decode a tokenised training sample and compare it byte for byte against the inference prompt.