Pick the right policy for stale tool outputs in a long agent trajectory

Long agent trajectories accumulate tool observations, web-search results, file reads, API responses, database queries, and each one can be hundreds or thousands of tokens. Across 20 or 30 steps, the raw observation history can easily blow past the effective context budget. The four options in the question span the design space, and three of them break in interesting ways. **Keep every tool output forever** sounds safe but bloats the context. The token budget is consumed by content the model has already extracted what it needs from, and the long history pushes the user's question and the current plan deep into the lost-in-the-middle trough. The model's accuracy drops even though it has "all the information." **Drop tool outputs immediately after the next model turn** is the opposite extreme. It keeps the context lean, but it loses information the agent does need to recall later. If step five asks about a number step two looked up, the answer is gone. **Move tool outputs to a system block at the top of the context** is structurally wrong on two axes: it sinks them into the middle of a long context as the trajectory grows, and (if the system block is supposed to be cached) destroys prompt caching. The right pattern is **summarize then evict**. Let the raw tool output sit in context just long enough for the next reasoning step to consume it. Once that step has produced its output, replace the raw observation with a compact summary or a pointer,"web_search returned 12 results; the relevant fact was X (call recall_observation(4) for full text)." The summary preserves the signal of the observation at a fraction of the cost, and the agent can recall the full raw text if a later step really needs it.

The correct option is **summarize then evict**, and the reason is that the design space splits cleanly into four positions and three of them have structural failures. **Keep everything (option 0)** trades short-term recall for long-trajectory collapse. Tool outputs are dense, and storing all of them indefinitely means a 30-step agent ends up with 30-50K tokens of historical observations in context. That weight pushes the user's question, the current plan, and the most recent reasoning into the lost-in-the-middle trough where attention is weakest. The agent has the data and cannot use it. The cost per call also scales linearly with trajectory length, which is the wrong shape for a long-running system. **Drop immediately (option 1)** trades long-trajectory collapse for catastrophic recall loss. The agent often needs to reason against a fact it learned several steps ago, to compare two retrieved figures, to revisit a search result, to refer back to an earlier file read. If the raw observations are gone, the agent has to redo the work, which is both slow and risks getting a different answer. **Top of context system block (option 3)** is broken on two axes. First, as the trajectory grows, content placed at the start sinks toward the middle of an ever-larger context window. The intuition that primacy is durably privileged is wrong; the start of context strong-attention region is small and fixed, while the middle grows around it. Second, if the system block is supposed to be cached for cost reasons, putting mutating tool outputs there destroys the byte-stable prefix and the cache rate drops to zero. **Summarize then evict (option 2)** is the working middle. The lifecycle is: the raw tool output enters context at the recency end (where the model can attend to it well), the immediate next reasoning step consumes it and produces its output, and then a separate process replaces the raw observation with a compact summary or pointer.