What is a stop sequence in an LLM API call and what is it used for?

Flashcard·Easy·4.0 · 0·~30s·Asked atDeepseek Lyzr·Relevant atAnthropic

Attempt it

A stop sequence is a string (or list of strings) you pass to the API. The model halts generation the moment it produces that string, and leaves the string out of the returned output. Common uses are enforcing structured output boundaries (stop on '</answer>' or '\n\nHuman:'), preventing the model from continuing past a known delimiter in a few-shot template, and terminating an agent loop on a sentinel token. The difference from max-tokens is the trigger. Max-tokens caps the reply at N tokens regardless of content (a length ceiling). A stop sequence terminates on a content match (a structural boundary). They are often used together. Max-tokens is the safety net, and stop sequences are the intent. A stop string can span several tokens, so test multi-token strings against the real API to avoid surprises.

Your answer

**A stop sequence** is a string you pass to the LLM (large language model) API. The moment the model generates that string during its reply, generation halts. The API strips the stop string from the returned output, so you get a clean response that ends right before it. Most APIs let you pass a list of stop sequences, not just one. The Anthropic, OpenAI, and Google APIs all support multiple stop strings, and the model halts on whichever one appears first. The most common uses are structural. If you are using a few-shot template (a prompt with worked examples) where each example ends with '\n\nHuman:', you set the stop sequence to '\n\nHuman:'. The model then writes one assistant turn and halts instead of inventing the next user message. If your prompt asks for JSON wrapped in <answer> tags, you can stop on '</answer>' to guarantee a clean cut. A stop sequence is a content boundary, not a length cap. It says terminate when this content appears. The max-tokens parameter says terminate after this many tokens regardless of content. They are different tools for different jobs and you usually set both.

**A stop sequence** is a content-based termination signal. It is a string (or list of strings) the model halts on the moment generation produces it. The stop string itself is excluded from the returned output. It is the cleanest way to enforce a structural boundary in model output. Three typical uses cover most production prompts. Boundary enforcement in few-shot templates: when your prompt has the pattern Human: ... Assistant: ... Human: ... Assistant: ..., set the stop sequence to '\n\nHuman:'. This guarantees the model writes one assistant turn and stops, rather than inventing the next user message. Format termination on tag-wrapped output: stopping on '</answer>' gives a deterministic end to a structured section. Agent-loop sentinels: a ReAct agent stops on '\nObservation:' so the orchestrator can run the tool and insert the real result. The critical distinction from **max-tokens** is the trigger condition. Max-tokens fires when the model has produced N output tokens regardless of what those tokens say (a length ceiling, blunt). Stop sequence fires when the running output matches the stop string (a structural boundary, precise). They are not alternatives. Production prompts set both. Stop sequences express intent, and max-tokens is the safety net in case the model never reaches a stop. One implementation caveat: a stop string can span several tokens. For multi-token strings, test against the real API to confirm the match works as expected.

TL;DR

A stop sequence is a string that halts generation the moment the model produces it. It enforces structural boundaries in output. Max-tokens is different: it caps length.

Memory aid

Easy to grasp

Imagine asking a friend to tell you a story but to stop the second they say the end. A stop sequence is that instruction for an LLM. You hand the API a string. The model writes one token at a time. After each step, it checks whether the running output now ends with your stop string. The moment it does, generation halts and that stop string is sliced off before the reply is returned. This is different from telling the friend stop after exactly two minutes, which is what max-tokens does. The stop sequence is a content rule. Max-tokens is a length rule. You usually want both. The stop sequence is the intent, and max-tokens is the safety net in case the model never reaches the stop string.

Key concepts

Stop sequence max-tokens Output control

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Stop sequences look like a trivial API parameter until you try to ship a structured-output prompt. Then you discover the model keeps generating past the boundary you wanted. The model does not natively know about your downstream parser. You have to tell it where to stop. The stop sequence is how you tell it.

What a stop sequence is and how it is enforced

A stop sequence is a string (or a list of strings) you pass to the LLM API alongside your prompt. During generation, the inference loop decodes one token at a time and appends it to the running output. After each step it checks whether the running output ends with any of the stop strings. On match, decoding halts immediately, the stop string is removed from the returned text, and the response is finalized.

The enforcement is at the inference level, not the prompt level. You can tell the model in the system prompt to stop when it produces some delimiter. But compliance with prose-level instructions is probabilistic. The stop sequence parameter is deterministic. The inference loop halts the moment the match condition is true, whether or not the model would have continued.

APIs vary in how many stop strings they accept. OpenAI's chat completions cap at 4 on models that support stop. Anthropic's API lists no count limit. Most providers treat the list as an OR: generation halts on whichever stop string appears first in the output. The order you pass them in does not matter. The first one the model produces wins.

Why max-tokens is not a substitute

The three production patterns

Provider-specific gotchas

Designing a good stop sequence

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Anthropic's Claude API accepts a stop_sequences array. The response payload reports stop_reason=stop_sequence when one fires, so clients can tell clean termination from length-cap termination.
OpenAI's chat completions API accepts up to 4 stop strings via the stop parameter on models that support it (stop is not supported with reasoning models such as o3 and o4-mini). The finish_reason field reports stop on a successful match.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat can go wrong when a stop string spans several tokens, and how do you test for it?

The model emits tokens, but a stop string is text, so it can begin or end inside a token. Hugging Face's StopStringCriteria exists to handle this case. Run the stop string through the target tokenizer and test it against the real endpoint.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Confusing stop sequences with max-tokens; one is a content boundary (terminate on string match), the other is a length cap (terminate after N tokens regardless of content).

60 second bullets to scan on the way to the call.

What a stop sequence is (string trigger that halts generation)
How it differs from max-tokens (content vs length trigger)

Primary sources. Browse if you want the original framing.