Click any words you think contain an error. Click again to unmark.
Gluing role labels to content with no spaces flips the boundary tokens to their no leading space variants, so the model sees off-distribution IDs; use apply_chat_template instead.
Imagine writing a play script where you forget to put a space between the character's name and their line, so it reads 'NARRATORonce upon a time'. A reader can sort of guess, but it looks wrong and they stumble. A tokenizer stumbles the same way. When you paste 'system:' straight onto the actual prompt with no gap, the first word of the prompt becomes a strange smushed-together piece the model rarely saw while learning. The fix is simple: add the spaces, or better, let the library format the chat for you.
Detailed answer & concept explanation~4 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: two boundary errors + leading-space token mechanism + why off-distribution at turn boundaries + local spaces fix + why it is insufficient + apply_chat_template + special tokens + model portability.
# Buggy: role labels glued to content, two broken boundaries
prompt = "system:" + system_prompt + "user:" + user_msg
tokens = tokenizer.encode(prompt) # first content word loses its leading-space token
# Local patch: restore boundary spaces
prompt = "system: " + system_prompt + " user:" + user_msg
# Real fix: let the model's own template format the chat
msgs = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_msg},
]
tokens = tokenizer.apply_chat_template(msgs, add_generation_prompt=True)Real products, models, and research that use this idea.
- HuggingFace's apply_chat_template() exists precisely to stop this bug, applying each model's Jinja template with correct spacing and special tokens for Llama 4, Mistral Large 3, and Qwen 3.5.
- Early LangChain and llama.cpp users hit garbled outputs from hand-concatenated role strings before chat-template support became standard.
- OpenAI's chat completions API hides this entirely by taking structured role messages, so callers never concatenate raw role labels for GPT-5.5.
- Anthropic's messages API for Claude Opus 4.7 likewise takes typed role turns rather than a single hand-built string, removing the boundary hazard.
What an interviewer would ask next. Try answering before peeking at the approach.
QEven with the spaces added, why is this prompt still wrong for a real chat model?
QHow would you detect this class of bug in an existing codebase before it ships?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Assuming string concatenation is tokenizer-neutral, so 'system:' + prompt tokenizes the same as 'system: ' + prompt.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.