Qwen 3.5 SFT data: fill the special tokens that wrap an assistant turn.

The two tokens are <|im_start|> and <|im_end|>. Qwen uses a grammar called ChatML for its chat templates. Each turn in a conversation is wrapped between two markers. The opening marker, <|im_start|>, tells the model a new turn is starting. Then comes the role name (system, user, or assistant) and a newline. Then the actual content. Then the closing marker, <|im_end|>, and another newline. For an assistant turn the full pattern looks like: <|im_start|>assistant\n{content}<|im_end|>\n. Both markers are single special tokens registered in the tokenizer's vocabulary. They are not regular strings that the tokenizer would break into smaller pieces. If you type them out as plain ASCII without telling the tokenizer they are special, you can end up with a model that sees a string of pipes and angle brackets instead of the actual stop signal. The reason this matters: if your SFT training data drops the <|im_end|> token from the labels, the model never sees its own stop signal during training. At inference it then refuses to stop where the chat template says it should, and you get rambling output.

**The two tokens are `<|im_start|>` and `<|im_end|>`.** Qwen 3.5 inherits the ChatML grammar. Every turn in a conversation is wrapped: ``` <|im_start|>{role}\n{content}<|im_end|>\n ``` For an assistant turn specifically the role is `assistant`, so the wrapped string is `<|im_start|>assistant\n{content}<|im_end|>\n`. Both markers are single registered vocabulary tokens, not multi-character strings the tokenizer would split into pieces. When you call `tokenizer.apply_chat_template(messages, tokenize=True)` against the official Qwen tokenizer, those special tokens render as single ids. If you handcraft training data as raw strings without going through apply_chat_template, you risk the tokenizer treating the literal characters as a sequence of normal tokens, and the model never sees the actual special-token ids. The practical danger is on the labels side. If you tokenize the prompt correctly with apply_chat_template but then slice the labels in a way that drops the trailing `<|im_end|>` (a common bug is `labels = input_ids[:-1]`), the model never learns to emit the stop token. At inference your stop rule fires on `<|im_end|>`, the model rarely produces one, and generation runs past the expected end of turn. The fix is to keep `<|im_end|>` as a supervised label in the response span and mask only the prompt positions to -100.