- 1User's original request restated
- 2Rolling summary of prior agent iterations
- 3Current plan and outstanding sub-goal
- 4Most recent tool call results, with stable identifiers
- 5Persistent memory facts about the user
- 6System prompt with persona, policies, and tool definitions
Order from most-cacheable and most-stable at the top, to most-recent and most task critical at the bottom, so the recency slot carries the live goal.
Picture a chef's workstation during dinner service. The recipe book and the kitchen rules sit on a shelf above the counter, they almost never move. The chef's notes about regular customers sit next to the book. Earlier in the night a sous-chef wrote a short summary of what dishes already went out. The most recent tickets from the printer are clipped right in front of the chef. Above the cutting board, written on a whiteboard, is the dish currently being plated. And taped to the front of the whiteboard is the order slip the customer wrote, restated so it never gets lost. The chef glances at the bottom of the station for what to do next, and the static reference sits up top.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
An agent loop sends a fresh prompt on every iteration. That prompt is not one blob, it is an ordered stack of named slots. The order is engineered, not aesthetic, and it follows from three pressures that 2026 production systems take seriously: prompt caching wants a long static prefix at the top, lost-in-the-middle wants critical content out of the middle, and recency bias wants the live goal at the bottom.
This deep dive walks each of the seven slots in order, explains the pressure that pins it where it is, and shows how the wrong order produces the failure modes that are common in shipped agents: cache misses, goal drift, stranded tool output, and persona dilution.
Mental model: static at the top, compressed in the middle, live at the bottom. The first dynamic byte is the cache boundary. The last byte is what the model thinks about hardest.
Why static slots go at the top
The cache boundary argument
Anthropic prompt caching, OpenAI cached prompts, and Gemini context caching all share the same shape: they cache a contiguous prefix from byte zero up to the first cache breakpoint. If you change any byte in that prefix, the cache invalidates. On a long-running agent loop that does 20 to 100 iterations per session, the difference between a cache hit and a cache miss is roughly 10x cost and 3x time to first token on the prefix.
This forces a clean rule: anything that does not change across iterations belongs above the cache boundary. The system prompt with persona, policies, and tool definitions is the obvious resident. Persistent memory facts about the user (preferences, profile, durable constraints) also live here when they are stable for the session.
Order within the static block
Within the static block, conventional order is: persona and policy first, then tool definitions, then persistent memory. Persona and policy define the agent's identity; tool definitions define its action space; memory facts personalize behavior. Placing memory last in the static block keeps it adjacent to the dynamic block, which simplifies reasoning about cache invalidation if memory ever does need to update mid-session.
Anti-pattern: the cache-killer
A common anti-pattern is putting a fresh timestamp, request ID, or session counter at the top of every prompt for logging. This single dynamic byte at position zero kills the entire cache. The fix is to put logging metadata in a separate field (the API headers, a non-prompt log line) or at the very bottom of the prompt where it cannot poison the prefix.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- LangGraph agents render typed state into a prompt with the system block first and the latest tool message last before the next model call.
- Claude Code (Anthropic) places tool definitions and policy at the top of every turn, with the user message and recent tool results near the bottom to exploit prompt caching and recency.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the assembly change when you add prompt caching specifically?
The cache boundary becomes a hard architectural line. Everything above it must be byte-stable across iterations; anything dynamic gets pushed below. This often forces persistent memory to be either fully above the boundary (if stable per session) or fully below (if it updates mid-session).
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Putting the user's question at the top because it is the request, then watching the model forget the goal under 5,000 tokens of tool output and summary.
60 second bullets to scan on the way to the call.
Why static content sits at the top of an agent prompt
How prompt caching defines the cache boundary
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.