What does the max_tokens parameter control in an LLM API call?

**max_tokens** caps the number of output tokens the model will produce in its reply. It does not limit the input length at all. The input is bounded by the model's context window, which is a separate (much larger) ceiling. The mechanism is simple. During generation, the model produces one output token at a time. After each token, the inference loop checks: have we produced max_tokens tokens yet? If yes, halt. The output stops wherever the count is hit, even if that is mid-sentence or mid-JSON. The response payload tells you whether the model stopped because of max_tokens (finish_reason=length on OpenAI, stop_reason=max_tokens on Anthropic) or for some other reason (a stop sequence matched, or the model produced its natural end of turn token). Production code logs this so you can spot prompts that are routinely getting truncated. Why you set it. max_tokens is the safety net that bounds both cost and latency. Without a cap, a misbehaving prompt can generate until the context window runs out, costing dollars per call. Setting max_tokens to a sensible ceiling (200 for short answers, 1,000 to 2,000 for long-form, 4,000+ for extraction tasks) protects against runaway generation.

**max_tokens** is an output-length cap. It bounds the number of tokens the model is allowed to generate in its reply. The model produces tokens one at a time, and the moment the count hits max_tokens, decoding halts. The critical distinction is from the context window. The context window caps the total tokens (input + output) the model can process in one call. max_tokens caps only the output slice. A 4K max_tokens setting on a 1M-token context window means the input can use up to 1M minus 4K tokens, and the model will produce at most 4K tokens of reply. Three practical consequences follow. First, hitting max_tokens often means mid-sentence truncation. The model does not gracefully wrap up when it sees the limit approaching; it just stops. The response payload reports finish_reason=length (or stop_reason=max_tokens, depending on provider), and production code should treat that as a quality signal indicating the prompt either asked for too much or the format is unstable. Second, max_tokens is a cost cap. Output tokens are typically 3 to 5 times more expensive than input tokens, so an uncapped reply can dominate the cost of a call. Setting max_tokens to the smallest value that fits the expected reply size is one of the easiest cost levers in production. Third, max_tokens and stop sequences are complementary. Stop sequences express where to halt by content; max_tokens expresses where to halt by length. Production prompts set both, with stop sequences as the intent and max_tokens as the safety net.