Where does an agent keep short-term state during a single task?

When an agent is solving one task, it needs to keep track of what just happened: what the user asked, which tool it just called, what the tool returned, what its current plan is. All of that information is the agent's **short-term memory**. The key thing to understand is where it lives. The model itself does not remember anything between calls. Each call to the model is a fresh inference. Short-term memory exists in the prompt the runtime assembles, or in a state object the runtime keeps in memory and injects into each call. Its size is bounded by the context window of the model: roughly a million tokens for frontier models in 2026, but in practice quality degrades well before that. When the task ends, the runtime usually throws the state away. The next task starts with an empty short-term memory. That is what distinguishes short-term from long-term memory. Long-term memory persists across tasks, usually in a vector store or database, and is retrieved when relevant. Short-term memory is a single-task scratchpad that resets at the end.

**Short-term memory** is the agent's working state during one task. It holds the conversation history with the user, the sequence of tool calls and their results, intermediate plans the agent has produced, and any scratchpad notes used during reasoning. It lives in two places that are really the same place from the model's view. Physically, the runtime keeps a state object: a list of messages, tool results, and metadata. Logically, on every model call the runtime serialises that state into the prompt the model actually sees. The model is stateless, so the prompt is the only carrier of memory from one step to the next. Its capacity is bounded by the context window. Frontier 2026 models offer windows of a million tokens or more, but quality drops well before the limit because attention is not uniform and long histories suffer from lost-in-the-middle and context-pollution effects. When the task ends, short-term memory is destroyed. The next task starts fresh, which is exactly what distinguishes it from long-term memory. Long-term memory persists across tasks (a vector store, a database, a profile file) and is retrieved when the agent needs it. Short-term memory is the per-task scratchpad; long-term memory is the cross-task archive.