Tool results go right before the assistant's next reasoning turn, at the recency end of context, which is exactly where the native tool-use message formats from Anthropic, OpenAI, and Google place them.
Think about a relay race where one runner hands a baton to the next. If the first runner drops the baton at the starting line and the second runner is at the finish, the handoff fails, the baton is too far away to be useful. A tool result is the baton, and the next model turn is the runner waiting for it. The handoff has to happen at the exact spot where the next runner is standing. Placing tool output at the top of a long conversation is like leaving the baton back at the starting line; the next runner cannot see it through everything that has happened in between. The native chat format for tool use is just a relay rule: hand the baton off right at the spot where the next leg of the race begins.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Tool-result placement is one of the cleanest applications of the lost-in-the-middle principle in context engineering. The question has a single correct answer, the recency end of context, immediately before the next assistant turn, and the major LLM providers have built that answer into their native message formats. The interesting part is why the answer is this and what goes wrong when teams deviate from it.
This deep dive walks through the empirical attention shape that motivates the placement, the way the native tool-use message formats encode it, the failure modes that show up when a custom prompt layer ignores the rule, and the cross-turn recall pattern for observations that need to be remembered beyond the immediate next step.
The U-shaped attention curve and why recency is durable
The lost-in-the-middle finding (Liu et al. 2023) is that LLM attention over a long context is U-shaped: strong at the start of the context, strong at the recency end, weak in the middle. The result has been replicated across every modern frontier model and across long-context benchmarks like RULER (2024) and BABILong (2024-2025). It is the load-bearing empirical fact behind a lot of context-engineering practice.
For tool-result placement, the U-shape has a sharper implication than it does for static content. Tool results are part of an unfolding trajectory: each new turn pushes the previous turns further from the recency end. Content placed at the absolute top stays at the top in absolute terms, but as the conversation grows, the top is no longer the start of a small context, it is the start of a much larger one, and the middle has grown around it. A tool result placed at position 0 of a 5-turn conversation might be 200 tokens from the recency end; the same result in a 30-turn conversation is 12,000 tokens from the recency end, deep in the trough.
Recency placement has the opposite property. A tool result placed adjacent to the most recent assistant turn stays adjacent as new turns appear, because the model always operates against the latest state. The placement is durably in the strong-attention zone, not just initially in it.
This is why the answer to "where do tool results go?" is recency, not "either end." The start of context primacy slot is real but small and fixed in size; it is the right place for the system prompt, but not the right place for per-turn content that needs to compete for attention against everything that comes after.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's Claude API uses a tool_use block from the assistant followed by a tool_result block carrying the output, immediately before the next assistant response.
- OpenAI's Responses API places tool_calls as assistant message parts with paired tool messages right after, threading them inline with the conversation.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is the right way to give the model access to old tool results from many turns ago?
Do not try to keep them in the prompt continuously. Store them in agent state (LangGraph typed state, a memory store, a vector index over past observations) and expose a recall_observation(id) or search_history(query) tool. The model fetches the old result into the recency end of context only when it actually needs it, and only the bytes it needs.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Pinning tool outputs at the top of the conversation in the belief that primacy gives them priority, in a long trajectory that just buries them in the middle.
60 second bullets to scan on the way to the call.
Where in the context should tool results be placed?
What is the U-shaped lost-in-the-middle attention curve?
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.