Place the scratchpad as the latest assistant turn at the recency end; each iteration, evict consumed tool results and stale reasoning, keep only the live plan and open sub-goals.
Think of a chef working through a complicated order. Next to the stove is a small notepad with what to cook next, the dish in progress, and ingredients still needed. The chef does not pin every chopped onion ticket from the last two hours to that pad. Those go in the bin once used. The pad stays small and current so a quick glance answers what comes next. An agent scratchpad works the same way. It sits where the model is about to read next, holds only the plan and live sub-goals, and drops anything already used. The kitchen receipts (full reasoning history, tool logs) belong in the back office, which for an agent is the telemetry system, not the prompt.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Agents read and write the same context window dozens of times across a single user request. That context is not a passive transcript, it is the only memory the model has. The scratchpad is the part of context the agent itself maintains and rewrites: a small block where the current plan, open sub-goals, and notes to self live.
Where that block sits and how it changes per iteration is the difference between an agent that completes ten-step tasks cleanly and one that ping-pongs across redundant tool calls because its own state has drifted out of view. This walkthrough covers placement, pruning, the separation from telemetry, and the production wiring most teams converge on.
Why placement matters
Modern long-context transformers exhibit a U-shaped attention curve: tokens near the start and end of the prompt are attended to more reliably than tokens in the middle. This is the lost-in-the-middle effect, and it shows up at every advertised context length.
The scratchpad's job is to be read by the next model step. If you pin it to the system prompt at the very top, it lives at the start of context end, fine when the conversation is short, but as the agent iterates the system prompt slides further from the recency end and the scratchpad gradually loses salience.
The robust placement is at the latest assistant turn, immediately above the next tool result and the next user/system message that triggers the next decision. In Anthropic and OpenAI native tool-use formats, this is exactly where the last assistant's reasoning lives. The format was designed around recency for the same reason: the model is generating from the bottom up.
In typed-state frameworks like LangGraph, the scratchpad lives in a structured state object and the prompt template renders it into the latest assistant turn each iteration. The model never sees a stale copy.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
from langgraph.graph import StateGraph
from typing import TypedDict, Annotated
class AgentState(TypedDict):
user_query: str
plan: list[str] # live, mutable
open_subgoals: list[str] # live, mutable
last_tool_result: str # only most recent
iteration: int
def prune_scratchpad(state: AgentState) -> AgentState:
# Drop completed sub-goals
state['open_subgoals'] = [g for g in state['open_subgoals'] if not g.startswith('[done]')]
# Keep only the latest tool result; previous ones already folded into plan
return state
def render_prompt(state: AgentState) -> str:
# Scratchpad sits at the recency end, just before the next decision
return (
f'PLAN:\n{chr(10).join(state["plan"])}\n\n'
f'OPEN SUBGOALS:\n{chr(10).join(state["open_subgoals"])}\n\n'
f'LATEST OBSERVATION:\n{state["last_tool_result"]}\n\n'
f'Next step?'
)Real products, models, and research that use this idea.
- LangGraph's typed state lets you declare a `scratchpad: str` field that the agent node rewrites each tick; checkpointers persist it without forcing it into the prompt verbatim.
- Claude Code maintains a planning block that gets rewritten across tool calls, old plan steps are crossed out and the live plan stays at the bottom of its working context.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide the watermark at which the scratchpad triggers compaction?
Pick a token threshold relative to the model's effective context (not its max), trigger a summarization node, and keep a small verbatim tail of the last two iterations to avoid sudden amnesia.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Carrying every prior reasoning trace and tool result forward as if the scratchpad were a transcript. Each iteration the prompt grows linearly and the live plan drowns.
60 second bullets to scan on the way to the call.
Why scratchpad placement matters in the context of recency bias
The distinction between working memory and a decision-trace log
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.