What are BOS, EOS, PAD, and role marker special tokens, and what happens when each is missing or duplicated?

Special tokens are vocabulary entries that act as structural signals rather than real words. Four of them show up constantly, and each has its own failure mode. **BOS** (beginning of sequence) is added at the very start of the input. The model was trained to predict the first real word given BOS as context, so it sets up the model's initial state. Drop it and the first prediction starts from an unexpected state, which usually nudges quality down a little. EOS (end of sequence) is the stop signal. During generation the model produces one token at a time, and when it produces EOS, generation halts. If the model never emits EOS, generation runs until a hard limit like `max_new_tokens`, so you get text that is cut off or rambles. PAD fills shorter sequences in a batch so every row has the same length. PAD carries no meaning, so you must hand the model an attention mask that says 'these positions are fake, do not look at them'. Role markers like `<|user|>` and `<|assistant|>` tell an instruction-tuned model who is speaking. Missing or wrong markers make the model treat a system instruction as user text or keep writing in the wrong voice.

There are four special token families, and the cleanest way to remember them is by their failure modes. - **BOS** (`<s>`, `<|begin_of_text|>`): marks sequence start. Missing it shifts the first-token distribution away from what training saw, so base-model quality drops slightly. - EOS (`</s>`, `<|eot_id|>`): the learned stop signal. Without it, sampling never halts on its own and runs to the `max_new_tokens` cutoff, giving truncated or repetitive output. - PAD: aligns variable-length sequences in a batch. It must be attention-masked, or the softmax assigns weight to empty positions and pollutes the value-weighted sum. - Role markers (`<|im_start|>`, `[INST]`, `<|user|>`): define conversation structure in instruction-tuned models. Misplaced markers break role-following and can leak the system prompt. The practical takeaway: do not hand-assemble these strings. Call `apply_chat_template()`, which inserts BOS, EOS, and the right role markers for the specific model you are serving.