Streaming tools force a decision the synchronous model dodges: when do you stop reading and feed what you have back to the model? Three patterns: (1) timeout-based, read for up to T seconds then truncate, (2) size-based, read until M tokens accumulate then return, (3) marker-based, read until the tool emits a sentinel. Each has failure modes. Timeouts cut mid-sentence; size limits truncate before completion; markers require tool cooperation. Pair the chosen policy with a 'continue' tool the model can call to resume reading. This pattern is now standard in coding agents like Claude Code and Cursor, where shell commands can stream for minutes.
Most tools in an agent loop are synchronous: the model calls, the tool runs, the result comes back complete, and the model reads it. Streaming tools break that assumption. A long-running shell command, a server sent events feed, a database query that streams rows, or a build process that produces output over minutes, all of these produce *partial* output continuously, and the agent has to decide when to stop reading and feed what it has back to the model.
There are three canonical stopping policies, and each one has a failure mode.
**Timeout-based.** Read for up to T seconds, then truncate whatever has arrived. Simple and predictable. The failure mode is that the cut can happen mid-sentence, mid-line, or in the middle of structured output, so the result is malformed and the model has to interpret a half-message.
**Size-based.** Read until M tokens have accumulated, then return. Predictable budget. The failure mode is that the cut happens before the tool has finished producing useful output, for instance, a build command might emit 5K tokens of progress lines before producing the one line that matters.
**Marker-based.** Read until the tool emits a sentinel, an explicit "done" marker, an empty line at the end of a structured block, or a known terminator. This gives clean stopping points. The failure mode is that the tool has to cooperate and emit the marker, which is often not under the agent's control.
In practice, production agent systems pair the chosen policy with a **continue** tool: the model can call `continue_stream(stream_id)` to resume reading where the cutoff happened. This is now standard in coding agents like Claude Code, Cursor, and Aider, where shell commands and test runs can stream for minutes and the agent has to read in chunks.
Streaming tools break the synchronous-tool assumption that the tool result is complete before the next model turn, and they force the wrapper to make a choice that synchronous tools dodge: when to stop reading and hand control back to the model.
The three canonical stopping criteria are timeout, size, and marker.
**Timeout** sets a wall-clock budget. Read for up to T seconds, then return whatever has accumulated, even if the stream is still active in the background. The advantage is predictable latency, which matters when the agent is in an interactive loop. The failure mode is that the cutoff has no relationship to content structure, the stream gets cut mid-line, mid-JSON-object, mid-sentence, and the model has to interpret a structurally broken last chunk.
**Size** sets a token budget. Read until M tokens have accumulated, then return. The advantage is predictable context cost. The failure mode is that important output may come late: a build process emits thousands of tokens of compile lines before the single "BUILD FAILED: missing import on line 42" message, and a size cap may cut just before the signal.
**Marker** waits for a structural sentinel. Read until the tool emits an explicit terminator, an empty line at the end of a logical block, a known end-token. The advantage is that cuts happen at clean boundaries, so the chunk handed to the model is structurally complete. The failure mode is that the tool has to cooperate and emit the marker, which is often not under the agent's control, a third-party shell command does not insert sentinels for you.
The production pattern is to layer these. A common shape is: read until either a marker is emitted, a size cap is hit, or a timeout fires, whichever comes first. The wrapper records which condition stopped the read so the model knows whether to expect more.
The load-bearing companion mechanism is a **continue tool**.
Quick reference
TL;DR
Streaming tools force the wrapper to choose when to stop reading: timeout, size, or sentinel, each with its own failure mode, and to pair the cutoff with a continuation tool the model can call to resume.
Imagine listening to a friend tell a story while you have a meeting in five minutes. You cannot make them stop mid-sentence and you also cannot wait for the ending. You have three reasonable choices. Set a timer and politely interrupt when it goes off. Decide ahead of time you will leave after a certain number of details. Or wait for a natural pause (the end of a chapter) and exit there. Each one cuts the story off in a different way, and the good move is to also promise your friend you will come back later and pick up where you left off. Streaming tools are exactly this: pick a stopping rule, accept that some endings will be jagged, and give the model a way to resume.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Tools that produce small, bounded output well under a second. If the underlying tool finishes fast and the output is small, treating it as a synchronous tool is simpler and avoids the overhead of streaming machinery. Streaming earns its complexity when the tool is genuinely long-running or produces large outputs.
Tools whose output is only useful in its entirety, like cryptographic operations or document generation. Partial output is meaningless or misleading for these tools, you cannot reason about half a signature or half a generated PDF. Treat them as synchronous and time out the entire call as one unit.
Single-shot non-interactive jobs where the agent has all the time it needs and the output fits in the budget. Batch pipelines without interactive loop pressure can just wait for the tool to finish and read the whole output. The streaming pattern earns its complexity when there is a model in the loop iteration.
Quick scan · 2–4 min
Everything important, quickly.
Define the problem: streaming tools break the synchronous-completion assumption. Name the three stopping criteria (timeout, size, marker) and one failure mode each. Describe the layered pattern: read until the first of marker / size / timeout fires, report which one stopped the read. Cover the continuation handle: stream_id parked in agent state, continue_stream and stop_stream as first-class tools. Touch on back-pressure (producer faster than consumer) and stall detection (producer stops emitting). Close with the 2026 production examples (Claude Code, Cursor, Aider) and the contrast with batch-style synchronous tools.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic's Claude Code bash tool streams shell-command output with a default timeout, a per-call token cap, and a 'truncated' marker plus a hint to run a narrower command.
2Cursor's terminal integration streams output to the agent in bounded chunks with the model able to issue a continue or kill operation.
3Aider streams build and test output back to its model with size-based chunking and a continue tool for resuming.
4OpenAI's Responses API exposes streamed tool outputs as tool_call_chunks, with the assistant deciding when to consume more.
5Devin and other autonomous coding agents stream long-running CI and deploy pipelines with timeout plus continue semantics so the agent can monitor without buffering the whole pipeline log.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you decide between buffering early stream content versus late stream content when truncating?
Sign in or go Pro to see the approach for this follow-up.
QWhat happens when the model wants to stop a stream early because it has already seen what it needs?
Sign in or go Pro to see the approach for this follow-up.
QHow do streaming tools interact with prompt caching across multiple continue calls?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating streaming as a synchronous tool by buffering the whole stream, the buffer either blows the budget or holds the agent for minutes; the policy choice is structural.