Bound each tool result at the tool boundary and stash overflow out of band, the bloat is input tokens from raw tool payloads, not anything the model is choosing to write.
Imagine a detective taking notes during interviews. After ten interviews, the notebook is so thick the detective can barely lift it, and most pages are coffee-shop menus and parking receipts the witnesses happened to be holding. The fix is not buying a bigger notebook. The fix is for the assistant who hands notes over to keep one short paragraph per interview and file the receipts in a drawer with a label. If the detective needs a receipt later, they can ask for it by label. That is what capping tool results does for an agent, the drawer is out of band storage, the label is the reference token in context.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Agent loops are deceptively expensive. Each iteration looks like a clean cycle, model plans, tool runs, result returns, model continues, but the context window is the accumulator, and most production loops have no garbage collection. A tool that returns 8KB of incidental data per call quietly grows the prompt by 2,000 tokens per iteration, billed and re-attended on every subsequent call in the trajectory.
This question is structural rather than tactical. Three of the four options treat the bloat as something to be managed at the model layer: enlarge the window, tighten temperature, ask politely for brevity. None of those touch the actual mechanism, because the model has no say in what its own inputs look like. The harness composes the prompt; the tool wrapper produces the payload. The fix has to live there.
The right answer is the one that addresses what the prompt is made of: cap each tool result at a known size, store overflow somewhere addressable, and let the agent pull more on demand.
The accumulator: what really happens per iteration
Take a search tool that returns the full HTML of the top result. The tool runs, the harness inserts the result into the assistant turn as a tool message, and the agent makes its next call. The model sees the full HTML on iteration 2. It also sees it on iteration 3. And on iteration 4. Each iteration carries every prior tool result forward, because the standard tool-use protocol (OpenAI, Anthropic, Vercel AI SDK, MCP) appends rather than replaces.
Why this hurts more than it looks
The damage is on three axes at once. Cost scales linearly in the cumulative tool-result size, each call pays the input rate on the running total. Latency grows the same way because time to first token is dominated by prompt length, especially for non-cached tokens. Quality degrades through the lost-in-the-middle effect: as the prompt grows, the user's actual task drifts toward the centre where attention is weakest.
A worked example
A ten-iteration trajectory with 8KB of tool exhaust per iteration accumulates around 20K tokens of low-signal input by the end. Multiply by 10 calls and you have paid input rate on roughly 110K tokens just to handle 80KB of source material the agent could have summarised once at 2K tokens.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
# Tool wrapper that caps and stores overflow out of band
from uuid import uuid4
MAX_TOOL_TOKENS = 1024 # roughly 1KB
def wrap_tool_result(raw: str, store) -> dict:
if len(raw) <= MAX_TOOL_TOKENS * 4: # ~4 chars/token
return {"kind": "inline", "content": raw}
head = raw[: MAX_TOOL_TOKENS * 2]
tail = raw[-MAX_TOOL_TOKENS * 2 :]
ref_id = uuid4().hex[:8]
store.put(ref_id, raw)
return {
"kind": "capped",
"head": head,
"tail": tail,
"elided_bytes": len(raw) - len(head) - len(tail),
"ref_id": ref_id,
"fetch_hint": f"call fetch_blob(ref_id='{ref_id}', offset=..., length=...) to read more",
}Real products, models, and research that use this idea.
- Claude Code caps shell command stdout at a budget and stores the full output, exposing a continue mechanism for the agent to read more.
- Cursor agent mode summarises long file reads into structured chunks before they reach the assistant message.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is a sensible eviction policy for the ids stashed out of band when the agent runs for hours?
Talk about reference counting versus time-based decay, and how summarisation can free ids whose content has been distilled into a memory block.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Asking the model to be brief, that controls output tokens; the bloat lives on the input side, where the model has no agency.
60 second bullets to scan on the way to the call.
What is the difference between input-side bloat and output-side bloat in an agent loop?
Where does a per-tool size cap belong: tool wrapper, system prompt, or model behaviour?
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.