Tool results go right before the assistant's next reasoning turn, at the recency end of context, which is exactly where the native tool-use message formats from Anthropic, OpenAI, and Google place them.
Think about a relay race where one runner hands a baton to the next. If the first runner drops the baton at the starting line and the second runner is at the finish, the handoff fails: the baton is too far away to be useful. A tool result is the baton, and the next model turn is the runner waiting for it. The handoff has to happen at the exact spot where the next runner is standing. Placing tool output at the top of a long conversation is like leaving the baton back at the starting line. The next runner cannot see it through everything that has happened in between. The native chat format for tool use is just a relay rule: hand the baton off right at the spot where the next leg of the race begins.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Tool-result placement is one of the cleanest applications of the lost-in-the-middle principle in context engineering. The correct placement is the recency end of context, immediately before the next assistant turn. The major LLM providers have built that answer into their native message formats.
The U-shaped accuracy curve and why recency is durable
The lost-in-the-middle finding (Liu et al. 2023) is that LLM performance over a long context is U-shaped by position. It is strong at the start of the context, strong at the recency end, and weak in the middle. The paper found the effect even in models built for long contexts. It is the key empirical fact behind a lot of context engineering practice.
For tool-result placement, the U-shape has a sharper implication than it does for static content. Tool results are part of an unfolding trajectory: each new turn pushes the previous turns further from the recency end. Content placed at the absolute top stays at the top in absolute terms. But as the conversation grows, the top is no longer the start of a small context. It is the start of a much larger one, and the middle has grown around it. At about 400 tokens per turn, a tool result placed just below the system prompt of a 5-turn conversation is about 2,000 tokens from the recency end. The same result in a 30-turn conversation is 12,000 tokens from the recency end. The system prompt holds the small start band, so the result now sits deep in the trough.
Recency placement has the opposite property. A tool result placed adjacent to the most recent assistant turn stays adjacent as new turns appear, because the model always operates against the latest state. The placement is durably in the high-accuracy zone, not just initially in it.
This is why tool results go at the recency end, not at "either end." The start of context primacy slot is real but small and fixed in size. It is the right place for the system prompt. It is not the right place for per-turn content that needs to compete for attention against everything that comes after.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's Claude API uses a tool_use block from the assistant followed by a tool_result block carrying the output, immediately before the next assistant response.
- OpenAI's Chat Completions API places tool_calls in an assistant message with paired tool messages right after; the Responses API uses adjacent function_call and function_call_output items.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat is the right way to give the model access to old tool results from many turns ago?
Do not try to keep them in the prompt continuously. Store them in agent state (LangGraph typed state, a memory store, a vector index over past observations) and expose a recall_observation(id) or search_history(query) tool. The model fetches the old result into the recency end of context only when it needs it, and only the bytes it needs.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Pinning tool outputs at the top of the conversation in the belief that primacy gives them priority. In a long trajectory, that buries them in the middle, far from the next reasoning step.
60 second bullets to scan on the way to the call.
Where in the context should tool results be placed?
What is the U-shaped lost-in-the-middle accuracy curve?
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.