How does the ChatGPT style cross-session memory differ from within session rolling summary?

Flashcard·Medium·4.0 · 0·~30s·Asked atGlean Lepton Ai Unity

Attempt it

Within-session rolling summary lives inside one conversation. Its job is to keep that conversation under budget as it grows: compress old turns into a running paragraph so the recent tail stays intact. It dies when the chat ends. ChatGPT-style persistent memory is a separate store of user-level facts (preferences, name, project context) that the system selectively injects into the context of future, unrelated conversations. Its job is cross-session continuity, not intra-session compression. The two coexist in a real product: persistent memory boots the new chat with the right user facts, and rolling summary keeps the chat coherent as it grows.

Your answer

These are two different memory systems that often get bundled into the same word "memory" but solve different problems. **Rolling summary** is **intra-session**. It exists to keep a single conversation under budget as it grows. The mechanism is to take older turns and compress them into a running summary paragraph (or hierarchical summary), so the recent verbatim tail stays intact while older context survives in compressed form. It is owned by the conversation and dies when the conversation ends. **Persistent memory (ChatGPT-style)** is **cross-session**. It is a separate store of user-level facts (name, preferences, project context, ongoing goals) that survives between conversations. ChatGPT shows this to the user as a manageable list of remembered facts; the system selectively injects relevant facts into the context of new, unrelated conversations. The store is owned by the user account, not by any single chat. The two coexist in a real product. Persistent memory boots a new chat with the right user facts ("the user prefers concise answers", "they are working on a project called Atlas"). Rolling summary keeps the chat coherent as it grows past the verbatim window. They have different lifecycles, different storage, different injection logic. The most common design mistake is treating them as the same thing.

**Rolling summary and persistent memory solve different problems with different lifecycles, and conflating them is one of the most common design mistakes in agent products.** **Rolling summary is intra-session compression.** A conversation grows; the verbatim turn budget is finite. Older turns get compressed into a running paragraph. The recent tail stays verbatim so pronouns, exact strings, and current intent survive; older arcs survive compressed. The summary is owned by the conversation, refreshed periodically, and discarded when the conversation ends. **Persistent memory is cross-session continuity.** A separate store holds user-level information: name, preferences, project context, goals. It survives across conversations and is owned by the user account. At the start of each new chat (or per turn), the system retrieves the relevant subset and injects it. **Why the distinction matters:** - **Different lifecycles.** Summary refreshes every N turns; persistent memory writes happen explicitly or via background extraction. - **Different storage.** Summary lives in conversation state; memory lives in a vector store (Mem0), graph (Zep), or typed store (Letta). - **Different retrieval.** Summary is always included; memory is selectively retrieved per turn. - **Different user surface.** ChatGPT shows memory as an editable list. Summary is invisible. **Coexistence in production.** A real chat product layers both. Persistent memory boots the new chat with personalisation context; rolling summary keeps the chat coherent past the verbatim window. Different slots, different policies.

TL;DR

Rolling summary is intra-session compression that dies with the chat; persistent memory is a cross-session user-fact store that boots every new chat with relevant context.

Memory aid

Easy to grasp

Imagine a long phone call with a friend versus a friendship across many phone calls. During one call you sometimes recap what you were just talking about so the call stays coherent. That recap dies when you hang up. Across many calls, your friend remembers things about you: where you live, what you do, what coffee you drink. They bring that up in future calls. Those are two completely different kinds of memory. One is about not losing track inside a single conversation. The other is about being a continuing relationship. A chat product needs both, and they live in different places.

Key concepts

Context engineering Agents Retrieval-augmented generation

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

"Memory" is one of those words that bundles several distinct primitives in a way that causes design problems. A real chat product has at least two of them, with different scopes, lifecycles, storage, and retrieval logic. The most common architectural mistake in 2026 agent products is treating them as one thing and stuffing them into the same slot.

This card walks through what each one actually is, why the frameworks of 2026 treat them as separate primitives, and what goes wrong when they get conflated.

Rolling summary: intra-session compression

Rolling summary exists to solve a specific problem: a conversation grows past the verbatim turn budget and you need to keep it coherent.

The pattern is straightforward. The most recent N turns stay verbatim (preserving pronouns, exact strings, current intent). Older turns get compressed into a running summary paragraph that occupies a fixed slot in the context. Periodically the summary is refreshed, usually every K turns, or whenever the verbatim slot would overflow.

Lifecycle. Born with the conversation. Refreshed during the conversation. Dies when the conversation ends. The summary state is part of the conversation state, not a global store.

Failure modes. Summary drift compounds when each refresh takes the previous summary as input instead of the original transcript. Periodic re-summarization from the raw transcript (or a windowed prefix of it) mitigates this. Summary noise is another failure: a too-aggressive compression loses pronoun referents or exact entity names that the model later needs.

Where it lives in the budget. A modest slot. For a 16k window, typically 800-1500 tokens. Its job is connective tissue, not main signal.

Persistent memory: cross-session continuity

Why the frameworks separate them

Failure modes from conflation

The 2026 production shape

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

ChatGPT's persistent memory feature shows users an editable list of facts the system has decided to remember across sessions; this is the canonical cross-session primitive.
LangChain's ConversationSummaryBufferMemory implements rolling summary as a within-session primitive; LangChain's separate store integrations handle persistent memory.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow do you decide what gets written to persistent memory versus discarded?

Two policies: explicit ('the user said remember X') and extraction-based (a background LLM pipeline pulls candidate facts from past conversations and scores them for durability). Filter on relevance, recency, and user-confirmation signal.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Conflating the two as 'memory' and putting them in the same slot. They solve different problems, have different lifecycles, and need different storage and retrieval logic.

60 second bullets to scan on the way to the call.

Define rolling summary in one sentence
Define persistent memory in one sentence

Primary sources. Browse if you want the original framing.