Flashcard: what is a stop sequence in an LLM API call and what is it used for?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

A stop sequence is a string (or list of strings) you pass to the API; the model halts generation the moment it produces that string, without including it in the returned output. Common uses are enforcing structured output boundaries (stop on '</answer>' or '\n\nHuman:'), preventing the model from continuing past a known delimiter in a few-shot template, and terminating an agent loop on a sentinel token. The difference from max-tokens is the trigger: max-tokens caps the reply at N tokens regardless of content (a length ceiling), while a stop sequence terminates on content match (a structural boundary). They are often used together: max-tokens is the safety net, stop sequences are the intent. Stop sequences are matched on token boundaries, not character boundaries, so multi-token strings work but you should test the exact tokenization to avoid surprises.

Your answer

**A stop sequence** is a string you pass to the LLM API as a hint: the moment the model generates that string during its reply, generation halts. The stop string itself is stripped from the returned output, so you get a clean response that ends right before it. Most APIs let you pass a list of stop sequences, not just one. Claude, OpenAI, and Google all support multiple stop strings, and the model halts on whichever one appears first. The most common uses are structural. If you are using a few-shot template where each example ends with '\n\nHuman:', you set the stop sequence to '\n\nHuman:' so the model writes one assistant turn and then halts instead of inventing the next user message. If your prompt asks for JSON wrapped in <answer> tags, you can stop on '</answer>' to guarantee a clean cut. The key thing to understand is that a stop sequence is a content boundary, not a length cap. It says terminate when this content appears. The max-tokens parameter says terminate after this many tokens regardless of content. They are different tools for different jobs and you usually set both.

**A stop sequence** is a content-based termination signal: a string (or list of strings) the model halts on the moment generation produces it, with the stop string itself excluded from the returned output. It is the cleanest way to enforce a structural boundary in model output. Three typical uses cover most production prompts. Boundary enforcement in few-shot templates: when your prompt has the pattern Human: ... Assistant: ... Human: ... Assistant: ..., setting the stop sequence to '\n\nHuman:' guarantees the model writes one assistant turn and stops, rather than inventing the next user message. Format termination on tag-wrapped output: stopping on '</answer>' or '```' or '---' gives a deterministic end to a structured section. Agent-loop sentinels: a ReAct or tool-using agent stops on 'Final Answer:' or '</tool>' to mark the end of one reasoning-action cycle. The critical distinction from **max-tokens** is the trigger condition. Max-tokens fires when the model has produced N output tokens regardless of what those tokens say (a length ceiling, blunt). Stop sequence fires when the running output matches the stop string (a structural boundary, precise). They are not alternatives; production prompts set both, with stop sequences expressing intent and max-tokens as the safety net in case the model never reaches a stop. One implementation caveat: stop sequences match on token boundaries in some APIs and on character boundaries in others. For multi-token strings, test the exact tokenization to confirm the match works as expected.

Quick reference

TL;DR

A stop sequence is a string the model halts generation on the moment it produces it, used to enforce structural boundaries in output, distinct from max-tokens which caps length.

Memory aid

Easy to grasp

Imagine asking a friend to tell you a story but to stop the second they say the end. A stop sequence is exactly that instruction for an LLM. You hand the API a string, and the model writes one token at a time, checking after each step whether the running output now ends with your stop string. The moment it does, generation halts and that stop string is sliced off before the reply is returned. This is different from telling the friend stop after exactly two minutes, which is what max-tokens does. The stop sequence is a content rule. Max-tokens is a length rule. You usually want both: the stop sequence is the intent, max-tokens is the safety net in case the model never reaches the stop string.

Key concepts

Stop sequence max-tokens Output control

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

6 min: define stop sequence as content-based termination, contrast with max-tokens, walk through three production uses, name provider differences in match semantics, point to stop_reason for debugging.

Real-world examples

Real products, models, and research that use this idea.

Anthropic's Claude API accepts a stop_sequences array; the response payload reports stop_reason=stop_sequence when one fires, letting clients distinguish clean termination from length-cap termination.
OpenAI's chat completions API accepts up to 4 stop strings via the stop parameter; the finish_reason field reports stop on a successful match.
LangChain's ReAct agent uses '\nObservation:' as a stop sequence to halt the model after each reasoning step so the orchestrator can inject the real tool output.
Many JSON-mode prompts use '```' as a stop sequence to terminate the fenced code block cleanly when the model is asked to produce JSON wrapped in a code fence.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does Anthropic match stop sequences on character boundaries while OpenAI matches on token boundaries, and what difference does it make?

QHow would you use the stop_reason field in production logging to catch failing prompts?

QWhat is the relationship between stop sequences, the end-of-turn token, and chat completion APIs?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Confusing stop sequences with max-tokens; one is a content boundary (terminate on string match), the other is a length cap (terminate after N tokens regardless of content).

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.