Sliding window is dead simple, deterministic, and zero-cost, but anything older than N turns is just gone, including durable facts the user mentioned early ('I am vegetarian', 'I work in finance'). Rolling summary preserves long-range information by compressing older turns into a running digest, but it pays an LLM-call cost on every summarization step and risks summary drift where details get smoothed away or hallucinated in. The pragmatic 2026 default is rolling summary plus an extracted-memory store: the summary handles continuity, the memory store handles durable facts.
Both patterns exist to bound the conversation-history tokens that get fed into the model, but they bound them in different ways and pay different costs.
**Sliding window** keeps the last N turns (or last K tokens) verbatim and drops everything older. It is the simplest possible policy: a FIFO queue over turns. The wins are concrete: zero LLM-call cost per turn, fully deterministic behavior, easy to reason about, easy to debug. The cost is also concrete: anything older than N turns is gone, including durable facts the user mentioned early like "I am vegetarian" or "I work in finance." The bot will silently behave as if those facts were never stated.
**Rolling summary** preserves long-range information by compressing older turns into a running digest, while keeping the last few turns verbatim. In principle nothing is ever fully lost: even early facts get folded into the summary. The cost is an LLM call to refresh the summary periodically, plus the risk of summary drift, where details get smoothed away or, worse, hallucinated into the summary.
In 2026, production stacks rarely pick one or the other. The pragmatic default is **rolling summary plus an extracted-memory store**. The summary handles narrative continuity, and the memory store handles durable facts the user wants preserved exactly.
Sliding window and rolling summary are the two basic shapes for bounding conversation-history tokens, and they trade off along the same axis: simplicity and determinism versus long-range information preservation.
**Sliding window** is a FIFO over the last N turns or last K tokens. Its wins are operational: zero LLM-call cost per turn, deterministic behavior, dead-simple to implement and debug. Its loss is structural: anything older than the window is gone. Durable facts the user stated early ("I am vegetarian," "I work in finance," their first name) silently fall off and the bot behaves as if they were never said. Latency spikes when the budget refreshes do not exist; cost is fixed per turn.
**Rolling summary** keeps older turns by compressing them into a running digest, then prepends the digest to the most recent verbatim turns. In principle, no information is lost forever, because every old turn passes through the summarizer before being dropped. In practice, prose compression loses exact numerics, rare named entities, long-distance pronoun antecedents, and verbatim artifacts like code blocks. The cost is an LLM call per refresh, plus the risk of drift across many refreshes (each refresh is a lossy encoding of an already lossy encoding).
The 2026 production default is to combine both with an extracted-memory store. The **sliding window** holds the most recent K turns verbatim, the **rolling summary** captures narrative continuity for older history, and the **extracted-memory store** carries durable facts as discrete items that survive across sessions. The three layers each handle what the others cannot: the window solves short-range antecedents, the summary maintains narrative arcs, and the memory store preserves the exact specifics that prose summarization always loses.
Quick reference
TL;DR
Sliding window is cheap and deterministic but drops everything past its cutoff; rolling summary keeps long-range continuity at LLM-call cost and risks detail drift.
Picture two ways to remember a long phone call with a friend. Option one: only remember the last five minutes word for word, forget anything before. Option two: keep a rough summary of the whole call updated in your head, plus the last five minutes word for word. Option one is fast and lazy but you lose anything the friend said early in the call, like the address they wanted you to visit. Option two preserves the big picture but the summary gets blurrier each time you update it, and exact details quietly disappear. Most production chatbots cheat: they run option two, plus they jot the address down separately in a notebook so it never gets lost.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Workloads where no fact older than a few turns is ever needed (single-task tools, transactional chat). Sliding window alone is sufficient and cheaper. Rolling summary adds LLM-call cost for no benefit.
Workloads where the entire conversation always fits comfortably in the model window. Neither technique is needed. Pass the full history verbatim.
Single-turn or stateless applications. There is no history to bound.
Quick scan · 2–4 min
Everything important, quickly.
Define both techniques in one sentence each. Walk through the trade-off: simplicity and determinism versus long-range preservation. Cover the failure modes of each. Explain the three-layer production composition. Close with the observability triage that maps failures to layers.
Side-by-side
Property
Sliding window
Rolling summary
Per-turn LLM cost
Zero (just generation)
Adds summarizer call on refresh
Determinism
Fully deterministic
Depends on summarizer model
Long-range info
Lost beyond N turns
Preserved at lower fidelity
Exact numerics
Preserved within window, lost outside
Drift on every refresh
Code/structured artifacts
Preserved within window, lost outside
Paraphrased away
Failure shape
Hard cutoff
Soft drift
Implementation complexity
Trivial
Moderate (prompt design matters)
Real-world examples
Real products, models, and research that use this idea.
1LangChain's ConversationBufferWindowMemory is the canonical sliding-window implementation.
2LangChain's ConversationSummaryBufferMemory is the canonical rolling-summary implementation.
3Mem0 is the canonical 2026 extracted-fact memory store, often paired with one of the above.
4Letta (formerly MemGPT) builds a hierarchical version of the rolling-summary idea with archival tiers.
5ChatGPT in 2026 combines a verbatim recent window, an internal summary, and a user-fact memory list.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QIf you can only pick one, which do you pick, and what is the failure mode you accept?
Sign in or go Pro to see the approach for this follow-up.
QHow do the three layers (window, summary, fact store) get tuned together?
Sign in or go Pro to see the approach for this follow-up.
QWhat does observability look like across the three layers?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating sliding window and rolling summary as mutually exclusive choices. The 2026 production default uses both, plus an extracted-fact store for durable specifics.