Spot the context engineering error in this tool result handling code

The snippet does two things that look reasonable on the surface and that interact catastrophically when combined. First, it appends every tool result to a `results_history` list inside the system prompt. The intuition is that the system prompt sits at the top of the context, in the privileged primacy slot, so tool outputs placed there should get strong attention. The intuition is wrong, because the system prompt grows monotonically with every tool call, and the rest of the conversation gets pushed deeper into the context, into the **lost-in-the-middle** trough. Second, it sends the same system prompt on every turn for prompt-caching benefits. The intuition here is also reasonable, caching is a real cost win, but it cannot coexist with a system block that mutates every turn. Prompt caches are keyed on byte-exact prefixes. The moment a new tool result is appended, the cache key shifts and the cache miss eats the cost. The combined effect is that the code pays full cost on every call (cache always misses) and also gets degraded quality (tool results in the primacy slot push real conversation into the trough). The correct pattern is opposite on both axes. Tool results live in the conversation message sequence, threaded between the assistant's tool call and its next reasoning turn. Stale results get evicted or summarized once they have been consumed by a downstream step. The system prompt stays byte-stable across turns and the cache actually hits.

Two compound errors. First, the snippet mutates the system prompt on every turn by appending tool results. This defeats prompt caching. Anthropic, OpenAI, and Google prompt caches are all keyed on the literal bytes of the prefix. A system block that grows with every tool call has a different byte sequence on every request, so the cache key never matches and the cache rate falls to zero. The line about "prompt-caching benefits" is exactly the part that does not work, the design promises caching while structurally making it impossible. Second, even setting caching aside, the system prompt is the wrong home for tool results. Tool results have a natural home in the conversation message sequence: the assistant's tool_use turn, then the tool_result turn, then the assistant's next reasoning turn. This is how Anthropic's tool-use format, OpenAI's tool_calls, and Google's function-calling format all encode it. Threading the result inline with the reasoning that consumes it places it at the recency end of the context, where attention is reliable. Putting tool results in the system block does the opposite. The system block is at the absolute start of context, which initially feels like a privileged position, but as more tool results pile up, the *non tool result* parts of the conversation (the user question, the assistant's plan, the earlier reasoning) slide deeper into the middle of an ever-longer context. The lost-in-the-middle effect, measured across all modern long-context models, says those middle tokens get attention-starved. A third issue worth naming is the lack of eviction. Tool results have a useful lifetime measured in steps, not in sessions. Once a step has consumed a tool result and produced its output, the raw result is dead weight. The correct pattern is summarize then evict: replace the raw result with a compact pointer ("web_search(query=X) returned 8 results; top three relevant to current step are ...") and free the budget for new observations. The