Sliding-window memory keeps only the last N turns (or last K tokens) of the conversation and drops everything older verbatim. It is cheap, predictable, and adds zero summarization cost per turn. The trade-off is hard: anything older than the window is gone. Long-range facts a user mentioned earlier (their name, a constraint set at the start, a decision made ten turns ago) silently fall off and the model behaves as if the user never said them.
**Sliding-window memory** is the simplest possible way to keep a chatbot's conversation history under a token budget. It keeps the last N turns (or the last K tokens) of the conversation and drops everything older. There is no compression, no summary, no replacement; older turns just stop being included in the prompt.
The mechanism is a FIFO queue over turns or tokens. When a new turn comes in, it goes to the front; if adding it would push the total over the budget, the oldest turn at the back falls off. The model only ever sees what is currently in the queue.
The wins are operational. It adds zero LLM-call cost per turn (no summarizer is run). It is deterministic: the same conversation history produces the same model input. It is dead-simple to implement (a list slice) and easy to debug (the model input is exactly the last N turns).
The one thing it cannot retain, by construction, is **anything older than the window**. A user who mentioned their name, set a constraint, or made a decision ten turns ago loses that information the moment the window slides past. The model then behaves as if that turn never existed. There is no graceful decay or partial preservation. The information is either present verbatim or completely absent.
**Sliding-window memory** keeps only the last N turns (or last K tokens) of the conversation verbatim and drops everything older. There is no summary, no extracted-fact store, no replacement of any kind. It is a FIFO over the conversation: turns enter at the front, fall off at the back when the budget is full.
The pattern is the simplest possible answer to bounded-context. The wins are operational and unambiguous. **Zero LLM-call cost** per turn beyond the main generation, because no summarizer is invoked. **Fully deterministic** behavior: the same conversation history produces the same model input on every call. **Trivial implementation**, often a one-line list slice. **Easy to debug**, since the model input is exactly the last N turns and nothing else.
The one thing it cannot retain, by construction, is **anything older than the window**. A user who told the bot at turn three "I am vegetarian," mentioned their company at turn five, or set a budget at turn seven loses each of those facts the moment the relevant turn falls off the back of the queue. The model does not approximate the missing fact, does not infer it from later context, and does not have any way to recover it. It simply behaves as if the turn never happened.
This is the **hard-cutoff failure mode**. Unlike rolling summarization, which degrades old content gracefully, sliding window is binary: present verbatim or completely absent. For applications where old facts never recur (single-task tools, transactional flows), the failure is benign. For chat assistants and agents where user-stated preferences might come up at any time, it is catastrophic.
The choice of N (or K tokens) trades reach against cost. Larger N keeps more context but eats budget. Production typical values land between 5 and 20 turns, depending on how far back pronouns and definite references typically point in the workload.
Quick reference
TL;DR
Sliding-window memory keeps only the last N turns verbatim and drops older ones. It is cheap and deterministic, but anything older than the window vanishes with no summary or substitute to replace it.
Imagine a fridge with a strict rule: only the five most recent items stay, anything older gets thrown out. No leftover gets summarized into a note on the door, no fridge inventory gets kept somewhere else. It is simple, the fridge never overflows, but the milk you bought last week is gone the moment the sixth item arrives. A sliding-window chat memory works the same way. The last few turns are kept word for word, anything older drops off the back, and nothing replaces them. The bot behaves as if those old turns never happened.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Long-running chat assistants where user-stated preferences might recur at any distance. Sliding window has a hard cutoff and will silently drop those preferences. Pair with a summary and an extracted-fact store.
Multi-session applications where context should persist across sessions. Sliding window has no notion of cross-session persistence. Use an extracted-fact memory store as the persistence layer.
Agents that need to recall decisions made early in a long trajectory. Once the trajectory exceeds N turns, the early decisions are gone. Layer with hierarchical summarization to keep them.
Quick scan · 2–4 min
Everything important, quickly.
Define the sliding-window pattern as a FIFO over turns or tokens. Walk through the wins (cost, determinism, debuggability). Walk through the one thing it loses (everything older than the window). Cover the choice of N or K and how to tune it. Close with the production composition where sliding window is paired with summary and fact-store layers.
Real-world examples
Real products, models, and research that use this idea.
1LangChain's ConversationBufferWindowMemory is the canonical sliding-window implementation.
2LlamaIndex's ChatSummaryMemoryBuffer falls back to a sliding window when summarization is disabled.
3Most chat templates in early prototypes use an implicit sliding window via 'last N messages' lists.
4Claude Opus 4.7 and GPT-5.5 APIs default to a sliding window style truncation if the client passes too many messages.
5Streaming chat interfaces in mobile apps often display only the last few turns visually; the underlying API is frequently a matching sliding window.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you pick the window size N for a chat application?
Sign in or go Pro to see the approach for this follow-up.
QWhen does a sliding window over tokens beat a sliding window over turns?
Sign in or go Pro to see the approach for this follow-up.
QHow does a sliding window interact with prompt-cache hits?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Thinking the dropped turns get summarized somewhere. In a pure sliding window, no summary is created and the dropped content is genuinely lost.