**Rolling summarization** replaces older turns with a running prose digest. It saves tokens, but the compression is not free, and the things it drops follow a predictable shape.
The first pattern is **numeric drift**. A user might say "my budget is 4,237 dollars" early on. When that turn rolls into the summary, the digest paraphrases it as "a budget around four thousand." Later replies use the round number, and the user notices the precise figure was lost.
The second pattern is **rare-entity loss**. Named entities that appear only once (a specific product code, an obscure place name, a colleague's name mentioned in passing) tend not to survive a summary refresh, because the summarizer optimizes for the gist, not for low-frequency tokens.
The third pattern is **broken pronoun chains**. If "she" referred to someone introduced ten turns ago, and the introduction got summarized away, the model now has "she" without a referent.
The fourth pattern is **paraphrased code**. The summarizer treats a pasted code block as prose to summarize, not as a verbatim artifact to preserve, so the user's code becomes a description of code.
Latency variability and faster first tokens are unrelated to summarization specifically; they happen for many reasons.
Rolling summarization compresses prior turns into a running digest. The compression is information-lossy, and the shape of that loss is predictable. Four failure modes are specifically diagnostic of over-reliance on this technique.
The first is **numeric drift**. Exact values, dates, and identifiers tend to round or vanish across summary refreshes. A user-stated 4,237-dollar budget shows up in a later reply as "about four thousand," and the user has to repeat themselves. Prose summaries optimize for gist; gist does not include four-digit precision.
The second is **rare-entity drop-out**. Named entities mentioned only once (a specific person, a less-common place, an obscure product code) are exactly the tokens a summarizer is most likely to elide. They contribute little to the gist of the conversation but are often the thing the user most cares about.
The third is **broken antecedent chains**. Pronouns and definite references ("she," "the file," "that decision") only resolve against the turn that introduced the referent. When that turn is summarized away, the pronoun in the live tail becomes a dangling reference, and the model is forced to guess or hallucinate.
The fourth is **verbatim artifact loss**. Code blocks, structured data, and exact-string payloads do not survive prose summarization. The summary becomes "the user pasted a Python script that does X," and the literal code is gone. If the user comes back to ask "can you modify the script I pasted earlier," the model can no longer see the script.
Latency observations like faster first tokens or variable per-turn cost are not summarization-specific failure shapes; they happen for many reasons in any dynamic-context system.
Quick reference
TL;DR
Rolling summarization predictably drops exact numerics, rare named entities, long-distance pronoun antecedents, and verbatim artifacts like code, because prose compression cannot preserve any of these by construction.
Imagine telling a friend about a movie you saw last month. You remember the gist, the main characters, the big twist. But you cannot recite the exact line the villain spoke, the address that was written on the note, or the phone number on the screen. Those details vanish the moment you summarize. A rolling summary in a chatbot has the same problem. It keeps the story but loses the exact numbers, the unusual names that came up once, the pronouns whose antecedents lived in the dropped turns, and the code blocks the user pasted. The bot starts sounding fluent but quietly forgetting the precise things the user actually cared about.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Sessions where exact numerics, codes, and identifiers are the primary content. Numeric drift will break the application. Use an extracted-fact store or verbatim retention instead of prose summarization.
Workflows where the user pastes code or structured data and references it later. Prose summarization paraphrases artifacts. The user's literal code disappears, and follow-up edits become impossible.
Short conversations that comfortably fit a sliding window. Summarization adds an LLM call cost per turn and introduces loss for no benefit when raw turns already fit in budget.
Quick scan · 2–4 min
Everything important, quickly.
Name the four loss patterns: numerics, rare entities, antecedents, artifacts. For each, explain the mechanism and a structural fix. Separate content-failure modes from latency observations. Close with the hybrid sliding-window plus summary plus fact-store pattern that production systems have converged on.
Real-world examples
Real products, models, and research that use this idea.
1LangChain's ConversationSummaryBufferMemory is the canonical rolling-summary implementation and exhibits these four loss patterns by default.
2Mem0 extracts facts into a separate store specifically to bypass numeric and entity drift in narrative summaries.
3ChatGPT memory in 2026 keeps a separate user-fact list precisely because prose summaries cannot preserve durable specifics.
4Claude projects expose a verbatim instructions block so code and exact strings are not paraphrased away by any internal summary step.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you detect numeric drift in production without manually reading transcripts?
Sign in or go Pro to see the approach for this follow-up.
QWhat does a hybrid sliding-window plus rolling-summary architecture look like, and which failure patterns does it solve?
Sign in or go Pro to see the approach for this follow-up.
QWhy might variable per-turn token usage look like a summarization failure even when it is not?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Assuming a rolling summary preserves everything except length. Prose compression always strips exact strings, rare entities, and verbatim artifacts, regardless of summary quality.