BOS starts generation, EOS stops it (infinite loop if missing), PAD aligns batches and must be attention-masked, and role markers fence off who speaks in a chat.
Imagine a stage play where the script has stage directions that nobody reads aloud. BOS is the 'curtain up, scene starts here' note. EOS is the 'curtain down, stop here' note, and without it the actor keeps improvising forever. PAD is blank filler added so every actor's script has the same number of pages, and the director has to remember not to read the blank pages out loud. Role markers are the name tags pinned on each speaker so the audience knows when the host, the guest, or the announcer is talking. Lose a name tag and the audience can no longer tell who said what.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Special tokens look like a beginner topic: four token types, four jobs, done. The reason they keep showing up in real incident reports is that each one fails quietly. A missing or misconfigured special token rarely throws an exception. Instead you get output that is slightly worse, a loop that burns your token budget, or a security hole that only an attacker notices.
This question also sits on the seam between tokenization and prompt engineering. People think of the prompt as the string they typed, but role markers and sequence boundaries are added by the tokenizer, not by the text. Knowing which layer owns each token is what lets you debug the failure instead of guessing.
The interview value here is that each token maps to a different part of the stack. BOS and EOS are about the generation loop, PAD is about batching and attention, and role markers are about conversation structure and security. A strong answer does not just name the four tokens; it explains the distinct failure each one causes and which layer you would inspect to fix it.
BOS and EOS: bookends of the sequence lifecycle
Every autoregressive sequence the model saw in training had the shape <BOS> token_1 ... token_n <EOS>. Two learned behaviors fall out of that shape.
The model carries a strong prior for the first real token given BOS. Remove BOS and the first prediction is conditioned on a zero-initialized or out of distribution starting state, which nudges quality down, most visibly on base models that were never wrapped in a chat template.
The model also learns a stopping criterion: the probability of EOS climbs as a response reaches a natural end. If EOS is absent from the sampling space, or the model never learned to emit it, generation runs to the max_new_tokens cap and stops mid-thought. The fix is rarely a longer cap. It is making sure EOS exists and the model was trained to produce it.
There is a practical wrinkle worth knowing. Some chat models define multiple end tokens, for example a generic end of text id plus a turn-level id like <|eot_id|>. If your stopping logic checks only one of them, the model may emit the other and keep going. This is why frameworks expose a list of stop ids rather than a single value, and why copying a generation config from a different model family is a common source of runaway output.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Llama 4 Maverick served locally needs the tokenizer's add_bos_token config set correctly, since base-model quality drops when BOS is missing from the prompt.
- OpenAI fine-tunes for GPT-5.5 use the ChatML format where <|im_start|>system and <|im_end|> fence each role, and malformed markers break the role contract.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the attention mask mechanically stop PAD tokens from earning attention weight?
Trace what happens to a PAD position's score before and after softmax once a large negative bias is added.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Saying PAD tokens are ignored automatically. They are not. You must pass an explicit attention mask, or the softmax spends weight on empty positions.
60 second bullets to scan on the way to the call.
The four special token families and what each one is for
Why a missing EOS produces an unbounded generation loop
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.