4Most recent tool-call results, with stable identifiers
5Persistent memory facts about the user
6System prompt with persona, policies, and tool definitions
An agent iteration is one trip through the loop: read context, decide, call a tool, fold the result back in, repeat. The prompt that gets sent on each iteration is assembled from several named slots, and the order of those slots is not arbitrary.
The top of the prompt holds the most static, most cacheable content. That is the **system prompt** with the persona, policies, and tool definitions, then any persistent memory facts about the user. These rarely change between iterations, which lets the prompt-caching machinery (Anthropic prompt caching, OpenAI cached prompts, Gemini context caching) keep the prefix warm and cheap.
The middle holds compressed history. A rolling summary of prior iterations sits here, replacing what would otherwise be a balloon of past observations. The most recent tool results come next, with stable identifiers so the model can cite them.
The bottom holds the live task. The current plan and outstanding sub-goal sit just above the user's restated original request. Recency dominates attention, so putting the goal last is what keeps the model on task across hundreds of tokens of intervening evidence.
The assembled prompt for one agent iteration is layered **static to dynamic, top to bottom**, with recency carrying the live goal.
Top layer (cached, rarely changes): the system prompt with persona, policies, and tool definitions, followed by persistent memory facts about the user. These bytes are identical across iterations within a session, which is exactly the shape that prompt-caching APIs reward.
Middle layer (compressed history): the rolling summary of prior agent iterations, then the most recent tool-call results with stable identifiers like `[T17]` or `tool_call_id`. The summary replaces older verbatim turns; the recent tool results stay verbatim because the model is about to reason over them.
Bottom layer (live task, gets the recency slot): the current plan and outstanding sub-goal, then the user's original request restated. The restatement is the single most common fix for long-context drift in agents. Models exhibit strong recency bias and the **lost-in-the-middle** curve buries content placed in the middle. Anchoring the goal at the bottom ensures the model's strongest attention is on what to do next.
This ordering also matches the typical cache-aware design: anything above the first dynamic insertion is cacheable; anything below is not. Putting the user's evolving plan at the bottom keeps the cacheable prefix maximal.
Quick reference
TL;DR
Order from most-cacheable and most-stable at the top, to most-recent and most task critical at the bottom, so the recency slot carries the live goal.
Picture a chef's workstation during dinner service. The recipe book and the kitchen rules sit on a shelf above the counter, they almost never move. The chef's notes about regular customers sit next to the book. Earlier in the night a sous-chef wrote a short summary of what dishes already went out. The most recent tickets from the printer are clipped right in front of the chef. Above the cutting board, written on a whiteboard, is the dish currently being plated. And taped to the front of the whiteboard is the order slip the customer wrote, restated so it never gets lost. The chef glances at the bottom of the station for what to do next, and the static reference sits up top.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Very short single-turn calls under 2K tokens with no tools The full slot ordering is overkill; a simple system + user prompt is enough and slot ceremony just adds noise.
Models known to disregard system-prompt position (some local fine-tunes) If the model card says it weights the last user turn dominantly regardless of position, slot order matters less than chat-template conformance.
Workflows with no persistent memory or rolling summary (one-shot tool use) Several middle slots are empty; collapse to system, tool result, user turn rather than maintaining empty slot scaffolding.
Quick scan · 2–4 min
Everything important, quickly.
5 to 7 min: walk the seven slots top to bottom, explain caching boundary, lost-in-the-middle, and recency, then show how the user-goal restate fixes drift.
Real-world examples
Real products, models, and research that use this idea.
1LangGraph agents render typed state into a prompt with the system block first and the latest tool message last before the next model call.
2Claude Code (Anthropic) places tool definitions and policy at the top of every turn, with the user message and recent tool results near the bottom to exploit prompt caching and recency.
3OpenAI Assistants API threads place run instructions at the top and the most recent tool outputs in the recency slot before each step.
4Cursor's agent mode restates the user's original ask in the recency slot on every iteration to prevent goal drift across long edit loops.
5Cognition's Devin and Replit Agent both publicly described keeping a 'goal restate' block at the recency end of every iteration.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the assembly change when you add prompt caching specifically?
Sign in or go Pro to see the approach for this follow-up.
QWhat changes for a multi-agent system where a sub-agent's output becomes the parent's tool result?
Sign in or go Pro to see the approach for this follow-up.
QHow do you handle the case where a tool result is itself longer than your soft budget?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Putting the user's question at the top because it is the request, then watching the model forget the goal under 5,000 tokens of tool output and summary.