Where does an agent keep short-term state during a single task?

Attempt it

Your answer

Quick reference

TL;DR

The agent's per-task working state (conversation, tool results, plans, scratchpad). It lives in the prompt or runtime state, is bounded by the context window, and resets at task end.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine someone solving a puzzle at a table. They keep a little notepad next to them with arrows and crossed-out guesses, the picture of the box, and a few tiles already placed. When the puzzle is done, they throw the notepad away and the next puzzle starts with a fresh notepad. Short-term memory is that notepad for an agent. It holds everything the agent needs to remember while working on this one task: what was said earlier in the conversation, what the last tool returned, what the agent is planning to do next. When the task ends, the notepad is gone.

Key concepts

Short-Term Memory Context Window Long-Term Memory

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

Define short-term memory as the per-task working state of the agent. Explain that the model is stateless and the runtime carries the state, materialised into the prompt each call. Name the contents: conversation, tool calls and results, plans, scratchpad. State the capacity bound (context window) and the effective bound (attention). Close with the contrast against long-term memory.

Real-world examples

Real products, models, and research that use this idea.

LangGraph's state graph keeps a typed state object per task; each node mutates the state and the runtime renders it into the prompt on each LLM call.
The OpenAI Agents SDK and Anthropic SDK both treat short-term memory as a per-session message list passed in on every call, with the runtime appending tool calls and results.
CrewAI agents share short-term memory inside a single crew run as a list of messages and intermediate outputs.
Cursor's coding agent maintains its short-term memory as the active conversation plus the open file contents, all serialised into the prompt each turn.
Claude Opus 4.7 and GPT-5.5 both expose large context windows, but agent runtimes still actively trim short-term memory to keep effective signal-to-noise high.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QIf frontier models have million-token context windows, why does effective short-term memory feel much smaller in practice?

Sign in or go Pro to see the approach for this follow-up.

QWhere exactly is the boundary between short-term and long-term memory in a system like LangGraph or the OpenAI Agents SDK?

Sign in or go Pro to see the approach for this follow-up.

QWhat happens if you try to carry the full short-term memory across task boundaries as 'session memory'?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Believing short-term memory is something the model 'has' the way humans have working memory. The model is stateless; short-term memory is just the prompt and runtime state assembled fresh on every call.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.