Spot the errors in this stop-sequence configuration recommendation

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

Quick reference

TL;DR

Stop sequences match the detokenized output string, not token IDs; tokenization is context-sensitive, so whitespace anchors matter and a wrong stop config plus no max_tokens means runaway generation and cost blowup.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine telling a dictation assistant to stop the moment it writes the word 'Stop'. You might assume 'Stop' is always the same secret code inside the machine. But the machine breaks words into chunks, and which chunk it uses depends on what came just before, a space, a newline, or the very start. So checking the secret code number is unreliable. Instead, the assistant should write out the actual letters and look at the visible text for the word 'Stop'. And if you never tell it a maximum length and the word never appears, it keeps writing forever, running up your bill. The safe setup matches the readable text and always caps the length.

Key concepts

Stop sequences Context-sensitive tokenization EOS token max_tokens cap

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

4 min: detokenized string matching versus token IDs + context-sensitive tokenization + whitespace anchoring + EOS versus stop strings + max_tokens as the runaway-generation circuit breaker.

Real-world examples

Real products, models, and research that use this idea.

vLLM and Hugging Face TGI both implement stop sequences as detokenized substring matching over a sliding window of recent output, not token-ID comparison.
The Anthropic Messages API for Claude Opus 4.7 returns stop_reason of 'end_turn', 'stop_sequence', or 'max_tokens', making the active terminator explicit per response.
OpenAI's GPT-5.5 chat completions expose both 'stop' strings and a max_completion_tokens cap, and runaway cost incidents trace to apps that set neither.
Llama 4 chat templates emit explicit turn-boundary tokens, and serving stacks add newline-anchored stop strings to avoid mid-turn false stops.
Postmortems from teams running DeepSeek V4 cite missing max_tokens caps as the root cause of overnight cost spikes when a fine-tune stopped emitting the expected EOS token.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy can a stop sequence that begins in the middle of a token never match cleanly?

Sign in or go Pro to see the approach for this follow-up.

QHow should a serving layer distinguish a genuine EOS stop from a stop_sequence stop from a max_tokens cutoff?

Sign in or go Pro to see the approach for this follow-up.

QWhat is the cost impact of omitting max_tokens on a high-traffic endpoint when the stop never fires?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Assuming a string maps to one stable token ID and matching on IDs, then shipping with no max_tokens. A missed stop with no cap means runaway generation and a cost blowup.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.