What does the max_tokens parameter control in an LLM API call?

The answer is C. **max_tokens** caps the number of output tokens (small chunks of text) the model will produce in its reply. It does not limit the input length (A) or input plus output (B); the model's context window, a separate and much larger ceiling, bounds those. It also does not set the size of the KV cache (the model's stored attention data), so D is wrong. During generation, the model produces one output token at a time. After each token, the inference loop checks whether it has produced max_tokens tokens yet. If yes, it halts. The output stops wherever the count is hit, even if that is mid-sentence or mid-JSON. The response payload tells you why the model stopped: finish_reason=length on OpenAI, stop_reason=max_tokens on Anthropic. Otherwise it names another reason, such as a matched stop sequence or a natural end of turn. Log it to spot prompts that often get truncated. Why you set it. max_tokens is the safety net that bounds both cost and latency. Without a cap, a misbehaving prompt can generate until it hits the model's output limit. A sensible ceiling (about 200 for short answers, 1,000 to 2,000 for long-form) protects against runaway generation.

The answer is C. **max_tokens** is an output-length cap. It bounds the number of tokens the model is allowed to generate in its reply. The model produces tokens one at a time, and the moment the count hits max_tokens, decoding halts. The key distinction is from the context window. The context window caps the total tokens (input + output) the model can process in one call. max_tokens caps only the output slice. A 4K max_tokens setting on a 1M-token context window means the input can use up to 1M minus 4K tokens. So A (prompt length) and B (input plus output) describe the context window, not max_tokens. D is wrong too: KV cache size depends on sequence length and model shape. Three practical consequences follow. First, hitting max_tokens often means mid-sentence truncation. The model does not wrap up when it sees the limit approaching; it just stops. The response reports finish_reason=length (or stop_reason=max_tokens, depending on provider). Treat that as a sign that the prompt asked for too much or the format is unstable. Second, max_tokens is a cost cap. Output tokens cost several times more than input tokens, and you pay only for tokens actually generated. The cap bounds the worst case: a runaway reply cannot dominate the bill. Third, max_tokens and stop sequences are complementary. Stop sequences halt by content; max_tokens halts by length. Production prompts set both, with stop sequences as the intent and max_tokens as the safety net.