Why does a streaming response that hits max_tokens cause problems for the next turn's context?

Attempt it

If you echo the truncated assistant turn back into context on the next call as-is, the model will interpret the half-finished sentence as a deliberate stylistic choice and either copy the truncation pattern or attempt to retroactively complete it in a way that contradicts what would have been generated. Three handling rules: detect the finish_reason of length and either (a) tag the truncated turn with an explicit marker that says generation was cut off, (b) replace the trailing partial sentence with an ellipsis at the last clean clause boundary, or (c) discard the partial generation and re-prompt with the same context plus an explicit instruction to be shorter. The cheap and safe default is (b).

Your answer

Quick reference

TL;DR

The model treats a mid-sentence echo as deliberate style.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine someone is telling a story and gets cut off mid-sentence by the school bell. If you write that half-sentence into the textbook word for word, the next reader assumes the storyteller chose to stop right there. They will either start ending their own sentences the same broken way to match the style, or they will try to finish the sentence in a way that does not match what was about to come. The fix is to either put a clear note that says cut off here, or trim back to the last finished sentence and add three dots, or throw away the half-sentence and ask the storyteller to tell a shorter version this time. The middle option is the easiest and works almost always.

Key concepts

Context engineering LLM frameworks Prompt engineering

Quick scan · 2–4 min

Everything important, quickly.

4 minutes: explain why echo-verbatim corrupts next-turn quality, name the finish_reason field across providers, walk the three handling strategies, and identify trim to ellipsis as the cheap and safe default for chat.

Side-by-side

Strategy	When to use	Cost	Drawback
Explicit marker	Agent loops with programmatic next-turn logic	~30 tokens overhead	Marker visible in chat UIs
Trim to clean boundary	Chat UIs with prose answers	Small information loss	User may want the partial content
Discard and retry	Structured outputs (JSON, tool calls)	Latency + double token spend	User waits longer for the answer
Echo verbatim (antipattern)	Never	Looks free	Model imitates the cutoff next turn

Real-world examples

Real products, models, and research that use this idea.

OpenAI GPT-5.5 returns finish_reason: 'length' on max_tokens hits; the OpenAI Python SDK exposes it on every response object.
Anthropic Claude Opus 4.7 returns stop_reason: 'max_tokens' versus 'end_turn'; the Anthropic SDK's Messages API surfaces this on every response.
Vercel AI SDK's streamText callback includes a finishReason field that fires on stream completion; production chat UIs branch on this to apply trim or retry logic.
Claude Code's task tool detects max_tokens cutoff and surfaces a 'response was truncated; retry?' UI rather than silently echoing the partial response.
LangSmith and Langfuse traces log finish_reason as a span attribute; teams use it to find 'high length cutoff rate' calls in production.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does Anthropic's prefill pattern interact with truncation handling?

Sign in or go Pro to see the approach for this follow-up.

QWhy does finish_reason: 'content_filter' need different handling than 'length'?

Sign in or go Pro to see the approach for this follow-up.

QHow do you instrument a production system to know when max_tokens is too low?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.