Keep the goal and the live plan; evict consumed tool results, dead-end branches, stale reasoning tokens, and resolved error messages from the agent's context.
Imagine a detective solving a case across many days. The whiteboard at the front of the room has the central question and the next move to investigate. The detective does not need to keep every dead-end interview transcript pinned up; once a lead is followed and concluded, the conclusion moves to the whiteboard and the transcript goes into a drawer. Without that discipline, the room fills with paper and the detective wastes hours rereading old leads. The same is true for an agent. The whiteboard is the live plan plus the user's question. Everything else is reference material that should be filed once its conclusion is on the board.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Agent loops are the failure mode that taught the field context engineering. Single-call prompts could be wasteful and the model would still answer; agents that run for ten or twenty iterations cannot afford waste, because the waste compounds.
This question is the canonical pruning decision: across one iteration's worth of work, which artifacts have done their job and can leave, and which must persist? Getting this wrong is the cheapest way to make an agent that gets dumber as it runs longer.
Why agent context is not append-only
The intuitive model of an agent is a journal: it remembers everything it did and refers back as needed. That model breaks for three reasons.
First, context windows are finite even when advertised limits are large. Frontier models advertise 200K to 2M tokens, but accuracy on retrieval and reasoning degrades well before those limits. An append-everything agent crosses the soft budget early in the trajectory.
Second, the lost-in-the-middle curve is a structural property of long contexts. Content in the middle of a long prompt is attended to less reliably than content at the start or end. An append-only trajectory steadily pushes the load-bearing items (current plan, latest observation) into the trough.
Third, prompt caching depends on a stable prefix. If the agent's context layout changes every iteration, the cache cannot amortize anything. Appending fresh content on top of the prefix is fine; threading new content into the middle of the prefix collapses the cache hit rate.
The alternative model is a working surface. Each iteration the surface holds exactly what the next call needs: the goal, the plan, the next step, and a small slice of recent observations. Everything else lives in an archive.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- LangGraph's typed state lets you declare which fields persist across nodes and which are scratch, making pruning a schema-level decision rather than a per-call cleanup.
- OpenAI's reasoning-summary mode for o-series models in 2026 stores intermediate reasoning server-side and exposes only summaries to subsequent turns.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide programmatically when a tool result is consumed and safe to evict?
Either annotate the tool result with the plan-item id it satisfied and evict when that item closes, or run a small classifier each iteration that marks observations as referenced or not based on the plan diff. The former is more reliable; the latter scales to less-structured agents.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating the agent's prior reasoning tokens as load-bearing context. The plan is load-bearing; the reasoning that produced it has done its job and can leave.
60 second bullets to scan on the way to the call.
Which categories of agent context have done their work after one iteration
Why the user's original request must persist across the whole trajectory
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.