How does streaming tool output complicate context budgeting in agent loops?

Most tools in an agent loop are synchronous: the model calls, the tool runs, the result comes back complete, and the model reads it. Streaming tools break that assumption. A long-running shell command, a server sent events feed, a database query that streams rows, or a build process that produces output over minutes, all of these produce *partial* output continuously, and the agent has to decide when to stop reading and feed what it has back to the model. There are three canonical stopping policies, and each one has a failure mode. **Timeout-based.** Read for up to T seconds, then truncate whatever has arrived. Simple and predictable. The failure mode is that the cut can happen mid-sentence, mid-line, or in the middle of structured output, so the result is malformed and the model has to interpret a half-message. **Size-based.** Read until M tokens have accumulated, then return. Predictable budget. The failure mode is that the cut happens before the tool has finished producing useful output, for instance, a build command might emit 5K tokens of progress lines before producing the one line that matters. **Marker-based.** Read until the tool emits a sentinel, an explicit "done" marker, an empty line at the end of a structured block, or a known terminator. This gives clean stopping points. The failure mode is that the tool has to cooperate and emit the marker, which is often not under the agent's control. In practice, production agent systems pair the chosen policy with a **continue** tool: the model can call `continue_stream(stream_id)` to resume reading where the cutoff happened. This is now standard in coding agents like Claude Code, Cursor, and Aider, where shell commands and test runs can stream for minutes and the agent has to read in chunks.

Streaming tools break the synchronous-tool assumption that the tool result is complete before the next model turn, and they force the wrapper to make a choice that synchronous tools dodge: when to stop reading and hand control back to the model. The three canonical stopping criteria are timeout, size, and marker. **Timeout** sets a wall-clock budget. Read for up to T seconds, then return whatever has accumulated, even if the stream is still active in the background. The advantage is predictable latency, which matters when the agent is in an interactive loop. The failure mode is that the cutoff has no relationship to content structure, the stream gets cut mid-line, mid-JSON-object, mid-sentence, and the model has to interpret a structurally broken last chunk. **Size** sets a token budget. Read until M tokens have accumulated, then return. The advantage is predictable context cost. The failure mode is that important output may come late: a build process emits thousands of tokens of compile lines before the single "BUILD FAILED: missing import on line 42" message, and a size cap may cut just before the signal. **Marker** waits for a structural sentinel. Read until the tool emits an explicit terminator, an empty line at the end of a logical block, a known end-token. The advantage is that cuts happen at clean boundaries, so the chunk handed to the model is structurally complete. The failure mode is that the tool has to cooperate and emit the marker, which is often not under the agent's control, a third-party shell command does not insert sentinels for you. The production pattern is to layer these. A common shape is: read until either a marker is emitted, a size cap is hit, or a timeout fires, whichever comes first. The wrapper records which condition stopped the read so the model knows whether to expect more. The load-bearing companion mechanism is a **continue tool**.