Explain the MemGPT paging metaphor and how Letta operationalizes it in 2026

Flashcard·Medium·4.0 · 0·~30s·Asked atAnthropic OpenAI Microsoft

Attempt it

The MemGPT paper (Packer et al. 2023) treated the context window as fast main memory and external stores as slow disk, with the model itself acting as the OS that decides when to page content in and out. Letta operationalizes this with named memory blocks (persona, human, archival) plus tools the model invokes to read, write, search, and edit those blocks. The model can grow a fact in long-term storage, evict an old fact when a block fills, and recall on demand via a search tool. The win is that memory management becomes part of the trajectory rather than an external policy applied by surrounding code.

Your answer

The **MemGPT paper** (Packer et al. 2023) borrowed a metaphor from operating systems: an LLM's context window is like fast main memory (RAM), external stores are like slow disk, and the model itself can act as the operating system that decides when to **page content in and out**. The concrete mechanism is **named memory blocks** plus **tool calls that edit them**. The original MemGPT design had three blocks: - **persona**: who the agent is, character, style, goals - **human**: who the user is, facts about them gathered over time - **archival**: a larger long-term store searched on demand The model interacts with these blocks through **explicit tools**: - `core_memory_append(block, content)`, add to persona or human - `core_memory_replace(block, old, new)`, edit a block - `archival_memory_insert(content)`, write a fact to archival - `archival_memory_search(query)`, search archival When a block fills up, the model decides what to evict, moving content to archival or rewriting the block to be more concise. When the model needs to remember something, it calls `archival_memory_search`. The decisions happen as **regular tool calls in the trajectory**, not as external framework policy. **Letta** (formerly MemGPT) is the 2026 production framework that operationalizes this design. It supports multiple backends, multiple agent types, and integrations with LangChain and other ecosystems. The named block plus tool pattern is the core abstraction. The win compared to a passive memory framework is that **the agent is the memory manager**. It learns when to recall, when to write, and when to evict as part of its trajectory.

The **MemGPT paging metaphor** reframed LLM memory as a **virtual-memory system** with the LLM itself playing the role of the operating system. The context window is fast main memory (limited, expensive, fast to access). External stores are slow disk (large, cheap, slow to access). The model decides what to keep in main memory, what to page to disk, and when to pull content back. This is a strong departure from the earlier pattern of treating memory as something external code did to the model. **The concrete mechanism is named memory blocks plus model-invoked tools.** MemGPT's design has memory blocks with explicit names and size limits: - **persona** (~1k tokens): who the agent is, character traits, behavioral style, capabilities, goals - **human** (~1k tokens): who the user is, gathered facts, preferences, ongoing situations - **archival** (unlimited, external vector store): the long-term store searched on demand - **recall** (recent conversation cache, automatic): a buffer of the most recent messages The model edits blocks through **explicit tool calls**: - `core_memory_append(block, content)`, add to persona or human - `core_memory_replace(block, old, new)`, surgical edit to a block - `archival_memory_insert(content)`, write a fact to archival - `archival_memory_search(query, top_k)`, semantic search of archival - `archival_memory_read(memory_id)`, read a specific archival entry When a core block approaches its size limit, the model receives a system warning and is expected to either rewrite the block more concisely or move content to archival. This is the **eviction policy operationalized as a trajectory decision**. **Letta is the 2026 production framework.** It is the rebrand and continued development of the MemGPT project. It supports multiple LLM backends (Claude, OpenAI, open-weights), persistent agents (state survives restarts), multi-user deployments, and integrations with the broader LangChain and ADK ecosystems.

Quick reference

TL;DR

MemGPT treats the context window as fast RAM and external stores as slow disk with the model as the OS; Letta exposes this through named memory blocks plus read, write, search, and edit tools the model invokes during

Memory aid

Easy to grasp

Imagine your desk holds only the papers you are actively using. Anything else lives in a filing cabinet across the room. A good worker pulls a file from the cabinet when they need it, puts notes back in the cabinet when they are done, and clears the desk when it gets cluttered. MemGPT proposes that an LLM should manage its own desk and cabinet system the same way: the desk is the context window, the cabinet is external storage, and the LLM decides what to pull, what to file, and what to throw away. Letta gives the LLM the actual tools to do this, open a drawer, write a note, search the cabinet, edit a tag.

Key concepts

Context Engineering LLM Frameworks

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Start with the MemGPT paging metaphor and what each role maps to (RAM, disk, OS). Describe the named memory blocks (persona, human, archival, recall) and their semantic roles. Walk through the tool operations the model invokes to read, write, search, and edit. Explain how block size limits surfaced to the model trigger eviction decisions. Identify Letta as the 2026 framework that productionizes this design, and close with the architectural payoff: memory management becomes part of the trajectory and is debuggable as a sequence of tool calls.

Real-world examples

Real products, models, and research that use this idea.

The original MemGPT research demo (2023) shipped agents that maintained user facts across long sessions by self-managed paging
Letta (2026) is used in autonomous-agent research and in some production assistants that need long-running memory
Anthropic's 2026 memory tool for Claude exposes a similar set of memory operations as tools the model invokes
LangGraph applications often borrow the named-block pattern, defining state slots with explicit semantic roles
OpenAI's GPT-5.5 Assistants API offers a memory feature inspired by similar model-driven recall patterns

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does Letta's design hold up when a single agent serves many users?

QWhat makes the surgical replace tool better than an overwrite tool for memory edits?

QWhy has the MemGPT framing influenced design beyond Letta itself?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating MemGPT as just a vector store with retrieval, missing that the key contribution is putting the model itself in charge of memory operations through explicit tool calls.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.