What failure mode occurs when a developer duplicates the BOS token by prepending it manually while also using apply_chat_template?

Most chat models expect their input to begin with one special **BOS token**, short for beginning-of-sequence. It marks 'the conversation starts here'. When you call `apply_chat_template(messages, add_special_tokens=True)`, the tokenizer already places that single BOS at the front for you. If the developer then prepends another BOS by hand, the sequence now starts with two BOS tokens instead of one. The problem is that the model was instruction-tuned only on sequences with exactly one BOS at the very start, immediately followed by the first role marker (like the system tag). It never saw two BOS tokens in a row during training. So every token gets shifted one position to the right, and the spot where the model expects the system role marker now holds a second BOS instead. The model can misread where the system instructions begin and end. In practice this means the system prompt may be partly or fully ignored, and the model can confuse which turn it is reading. The fix is simple: do not add BOS yourself. Let the template handle it.

The failure is a **double-BOS sequence** the model never trained on, and it corrupts role parsing. The call `apply_chat_template(..., add_special_tokens=True)` already inserts one BOS at position 0, which is exactly what the model expects. Prepending `tokenizer.bos_token_id` manually adds a second one, so the sequence opens with two BOS tokens. Why that hurts: the model was instruction-tuned on sequences with one BOS at position 0 followed immediately by the first role marker, often the system tag. With a duplicate, every later token shifts right by one. The position where the model expects the system marker now holds a second BOS, and the structural signal it learned to read is broken. The consequences are concrete. The system prompt may be misinterpreted, since attention at the system marker now points at an unexpected token. Role confusion appears, because the model cannot cleanly tell where the system turn ends and the user turn begins. In bad cases the system prompt is partially or fully ignored. One context position is also wasted, which is minor next to the structural damage. The fix has two forms. Either set `add_special_tokens=False` when adding BOS yourself, or better, pass `add_generation_prompt=True` and let the template own all special tokens.