Spot the tokenization boundary bug in this manual prompt-building code.

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

Quick reference

TL;DR

Gluing role labels to content with no spaces flips the boundary tokens to their no leading space variants, so the model sees off-distribution IDs; use apply_chat_template instead.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine writing a play script where you forget to put a space between the character's name and their line, so it reads 'NARRATORonce upon a time'. A reader can sort of guess, but it looks wrong and they stumble. A tokenizer stumbles the same way. When you paste 'system:' straight onto the actual prompt with no gap, the first word of the prompt becomes a strange smushed-together piece the model rarely saw while learning. The fix is simple: add the spaces, or better, let the library format the chat for you.

Key concepts

Chat templates Byte Pair Encoding

Quick scan · 2–4 min

Everything important, quickly.

4 min: two boundary errors + leading-space token mechanism + why off-distribution at turn boundaries + local spaces fix + why it is insufficient + apply_chat_template + special tokens + model portability.

Code example

python

# Buggy: role labels glued to content, two broken boundaries
prompt = "system:" + system_prompt + "user:" + user_msg
tokens = tokenizer.encode(prompt)  # first content word loses its leading-space token

# Local patch: restore boundary spaces
prompt = "system: " + system_prompt + " user:" + user_msg

# Real fix: let the model's own template format the chat
msgs = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_msg},
]
tokens = tokenizer.apply_chat_template(msgs, add_generation_prompt=True)

Real-world examples

Real products, models, and research that use this idea.

HuggingFace's apply_chat_template() exists precisely to stop this bug, applying each model's Jinja template with correct spacing and special tokens for Llama 4, Mistral Large 3, and Qwen 3.5.
Early LangChain and llama.cpp users hit garbled outputs from hand-concatenated role strings before chat-template support became standard.
OpenAI's chat completions API hides this entirely by taking structured role messages, so callers never concatenate raw role labels for GPT-5.5.
Anthropic's messages API for Claude Opus 4.7 likewise takes typed role turns rather than a single hand-built string, removing the boundary hazard.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QEven with the spaces added, why is this prompt still wrong for a real chat model?

Sign in or go Pro to see the approach for this follow-up.

QHow would you detect this class of bug in an existing codebase before it ships?

Sign in or go Pro to see the approach for this follow-up.

QYour team supports five different open-weight models. What is the single safest formatting strategy?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.