An agent loop is a cycle of model call, tool call, tool result, model call, and so on. Every iteration appends the new tool result to the running context. If a tool returns a large blob, say an 80KB HTML page or a 5MB log file, that blob now rides along on every future call in the same trajectory.
The arithmetic is harsh. Eight kilobytes per iteration is roughly 2,000 tokens. After ten iterations the context has grown by 20,000 tokens of pure tool exhaust. You pay for those tokens on every subsequent call, and the model now has to look past all of that to find the user's actual question.
The fix lives at the tool boundary, not inside the model. The tool wrapper parses the raw result, extracts what matters, caps it at a known budget like 1KB, and writes the full blob to an out of band store. What goes into context is a short structured object plus an identifier the agent can use later if it needs the rest.
Bloat at the rate of 8KB per iteration is a classic tool result drowning failure. The fix is structural, not behavioural, the model has no say in how big its inputs are.
Put a hard cap on every tool result at the tool wrapper layer. A search tool returns titles, snippets, and URLs, not raw HTML. A file-read tool returns the first and last N lines plus a length indicator, not the whole file. A shell tool truncates stdout at a token budget and notes how much was elided. The raw payload gets written to an out of band store keyed by a short id, and the id rides along in context. If the agent decides it actually needs the rest, it calls a `fetch_by_id` tool to pull the next slice.
The trap to avoid is reaching for a bigger context window. A 1M-token window does not make 200K tokens of HTML useful, it just makes the bill 5x bigger and the lost-in-the-middle problem worse. Lowering temperature and asking for brevity are unrelated; both affect output, and the bloat is input.
Quick reference
TL;DR
Bound each tool result at the tool boundary and stash overflow out of band, the bloat is input tokens from raw tool payloads, not anything the model is choosing to write.
Imagine a detective taking notes during interviews. After ten interviews, the notebook is so thick the detective can barely lift it, and most pages are coffee-shop menus and parking receipts the witnesses happened to be holding. The fix is not buying a bigger notebook. The fix is for the assistant who hands notes over to keep one short paragraph per interview and file the receipts in a drawer with a label. If the detective needs a receipt later, they can ask for it by label. That is what capping tool results does for an agent, the drawer is out of band storage, the label is the reference token in context.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
A one-shot agent that runs for at most two or three tool calls and then exits The bloat is unbounded only across many iterations; a short trajectory does not need the out of band machinery.
Tools that already return small, structured outputs by design (e.g., a calculator or a lookup that returns a single row) Capping is harmless but unnecessary when the natural output already fits the budget.
Cases where the model genuinely needs the full raw payload to reason, for instance, exact-byte log analysis Capping with summary would lose the signal the task requires; instead, page the payload deliberately or move the work to a deterministic tool.
Quick scan · 2–4 min
Everything important, quickly.
8 min: per-iteration growth math, the tool boundary fix, why bigger windows don't help, eviction and prompt-cache interactions.
Code example
python
# Tool wrapper that caps and stores overflow out of band
from uuid import uuid4
MAX_TOOL_TOKENS = 1024 # roughly 1KB
def wrap_tool_result(raw: str, store) -> dict:
if len(raw) <= MAX_TOOL_TOKENS * 4: # ~4 chars/token
return {"kind": "inline", "content": raw}
head = raw[: MAX_TOOL_TOKENS * 2]
tail = raw[-MAX_TOOL_TOKENS * 2 :]
ref_id = uuid4().hex[:8]
store.put(ref_id, raw)
return {
"kind": "capped",
"head": head,
"tail": tail,
"elided_bytes": len(raw) - len(head) - len(tail),
"ref_id": ref_id,
"fetch_hint": f"call fetch_blob(ref_id='{ref_id}', offset=..., length=...) to read more",
}
Real-world examples
Real products, models, and research that use this idea.
1Claude Code caps shell command stdout at a budget and stores the full output, exposing a continue mechanism for the agent to read more.
2Cursor agent mode summarises long file reads into structured chunks before they reach the assistant message.
3LangGraph's state object supports message trimming so tool results can be reduced before the next model call.
4The Model Context Protocol (MCP) standardises resource references so a tool result can return an id instead of a payload.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is a sensible eviction policy for the ids stashed out of band when the agent runs for hours?
Sign in or go Pro to see the approach for this follow-up.
QHow does this approach interact with prompt caching?
Sign in or go Pro to see the approach for this follow-up.
QIf you had to debug an agent that still bloats despite caps, where would you look?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Asking the model to be brief, that controls output tokens; the bloat lives on the input side, where the model has no agency.