max_tokens vs stop: pair each parameter with the cap it enforces on the completion.

Match pairs·Easy·4.0 · 0·~2 min·Asked atInflection Ai Infosys Qualcomm·Relevant atOpenAI

Attempt it

Drag each answer to line up with its matching prompt

max_tokens

max_tokens, the cap is enforced unconditionally on every request.

stop

stop, termination only fires if the model happens to emit a matching string.

Always applies

A list of string sentinels; generation halts when one is detected in the streamed output.

Content conditional

A hard upper bound on how many completion tokens the server will generate before halting.

When you call an LLM API, you usually want two kinds of termination control. The first is a safety cap: stop after some maximum number of generated tokens no matter what. The second is a semantic cap: stop when the model finishes saying a particular thing. **`max_tokens`** is the safety cap. You pass an integer like `max_tokens=500`, and the server commits to generating at most 500 completion tokens before halting. This is an absolute ceiling. It does not matter what the model is saying when it hits the limit. Generation stops, the partial output is returned, and the response is flagged with `finish_reason: length` so your code knows the answer is truncated. **`stop`** is the semantic cap. You pass a list of strings, like `stop=['\n\nHuman:', '###']`. The server watches the output as it streams and the moment it produces any of those strings, decoding halts. The matched string itself is NOT included in the returned output. If the model never emits any of the stop strings, this parameter never fires, and decoding continues until `max_tokens` or the context limit. In practice you use both. `max_tokens` bounds your worst-case spend. `stop` gives the model a clean off-ramp so it can end at the right narrative or structural boundary instead of getting cut off mid-word.

**`max_tokens`** is an unconditional integer cap on completion length. The server keeps a running token count and halts the decode loop the instant that count is reached. The response carries `finish_reason: length` so the client knows it was truncated rather than completed. **`stop`** is a conditional list of string sentinels. As tokens stream out of the decoder, the server scans the rolling buffer for any of the supplied strings. The moment one matches, generation stops. Critically, the matched sentinel is stripped from the returned output, so the response ends just before the stop string. If no sentinel ever appears, `stop` has no effect and decoding runs until some other limit binds. Production code uses both, for different reasons. `max_tokens` is a financial guardrail: every API call has a known maximum cost, which is essential for billing predictability and abuse mitigation. `stop` is a structural guardrail: it lets the model end naturally at a meaningful boundary, like the end of a JSON object, a turn marker in a chat template, or a delimiter in a structured prompt. The failure mode to avoid is relying on `stop` alone. If the model never emits your sentinel string, generation continues until `max_tokens` or the model's context limit, which can be tens of thousands of tokens and dollars of cost per call. Always set `max_tokens` even when you have a tight `stop` list.

TL;DR

`max_tokens` is a hard numeric ceiling that always fires; `stop` is a list of strings that only halts generation when the model happens to emit one.

Memory aid

Easy to grasp

Picture two ways to tell a storyteller when to stop. The first way is a kitchen timer. You set it to 60 seconds, and no matter what the storyteller is saying when it dings, they have to put down the microphone. That is `max_tokens`. The second way is a code word. You say 'when you reach the end of the story, say THE END, and we will know to clap.' If the storyteller never says the code word, you keep listening forever. That is `stop`. Production systems use both. The timer protects you from an endless tale that runs up the bill. The code word lets the storyteller end naturally at the right narrative beat instead of getting cut off mid-sentence by the timer.

Key concepts

max_tokens parameter Stop sequences Decode phase Structured output

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Every LLM API has two completely different ways to tell the model when to stop generating, and they are not interchangeable. max_tokens is a counter. stop is a pattern matcher. One is a hard cost guarantee. The other is a soft structural hint. Mixing them up is one of the most common production bugs in early LLM applications: a runaway request that costs a hundred times what the developer expected, simply because they set stop and assumed it would bound their bill.

This deep dive walks through what each parameter actually does inside the server, why the matched stop string is stripped from the response, how streaming layers handle multi-token sentinels without leaking partial matches, and the design rule that production systems always need both parameters set together. By the end, you should be able to read any LLM client snippet and predict exactly when and why decoding will halt.

max_tokens: the unconditional counter

max_tokens is the simpler of the two. You pass an integer, and the inference server keeps a counter incremented on every decode step. The instant the counter reaches the supplied number, the decode loop exits, the partial output is returned, and the response is flagged with finish_reason: 'length' (OpenAI) or stop_reason: 'max_tokens' (Anthropic) so the client knows the answer was truncated rather than completed.

What makes this parameter useful is that it is unconditional. The model has no say in it. No prompt engineering, no jailbreak, no stochastic sampling artifact can make decoding continue past the limit. From a cost and latency perspective, that determinism is the entire point. Every API call has a known maximum number of completion tokens, and therefore a known maximum dollar cost (input tokens are already fixed at submission time). You can do capacity planning, set budget alerts, and write SLAs against this.

The rule is to size max_tokens to your realistic worst case for the task plus some safety margin. Too low and complex answers get truncated mid-thought; too high and your worst-case cost ceiling balloons. A common pattern is to set it per task category: 200 tokens for classification labels, 1500 for code snippets, 4000 for long-form drafting. Some providers also cap the total prompt_tokens + max_tokens at the model's context window, so you cannot set max_tokens = context_limit if your prompt is non-trivial.

stop: the conditional pattern matcher

Streaming and the tail-buffer problem

The always-set-max_tokens rule

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Aspect	max_tokens	stop
Type	Integer	List of strings
Trigger condition	Unconditional, count reached	Conditional, string matched
Cost guarantee	Yes, hard ceiling	No, depends on model output
Output includes trigger	N/A, counts tokens	No, sentinel is stripped
Primary use	Bound spend and latency	Shape structured boundaries

Real products, models, and research that use this idea.

OpenAI's `/v1/chat/completions` endpoint accepts both `max_tokens` and `stop` (up to 4 strings); responses include `finish_reason: 'length'` or `'stop'` to indicate which fired.
Anthropic's Messages API uses `max_tokens` (required) and `stop_sequences`; responses include `stop_reason: 'max_tokens'` or `'stop_sequence'`.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does `stop` work with multi-token strings in a streaming response?

The server must keep a small tail buffer of recently generated text, equal to at least the longest stop string. It only flushes chunks once it can prove no stop string spans the boundary. This is why some providers cap stop strings at a few characters or a small count.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Relying on `stop` alone to bound cost. If the model never emits the sentinel string, decoding runs until the model's context limit, which can cost dollars per request.

60 second bullets to scan on the way to the call.

Definition of max_tokens as an unconditional integer cap
Definition of stop as a conditional list of string sentinels

Primary sources. Browse if you want the original framing.