How is long-term memory typically implemented in an agent?

Flashcard·Easy·4.0 · 0·~30s

Attempt it

Long-term memory persists across tasks: user preferences, prior conversation summaries, learned facts, past trajectory reflections. It is usually backed by a vector database (Pinecone, Chroma, pgvector, Weaviate, Qdrant) or a key-value store; the agent retrieves relevant entries each task and injects them into the prompt. It differs from short-term memory (in-context) and from model weights (frozen at inference).

Your answer

**Long-term memory** is the part of an agent's memory that survives across tasks. Short-term memory holds what is happening right now and resets when the task ends. Long-term memory holds things the agent should remember next time: who the user is, what they prefer, what was discussed last week, what worked and what failed. It almost never lives in the model. The model's weights are frozen during inference; nothing the user says updates them. Instead, long-term memory lives in an external store that the runtime owns. The two most common shapes are a vector database (Pinecone, Chroma, pgvector, Weaviate, Qdrant) for fuzzy semantic retrieval and a key-value or relational store for exact lookups like user profiles. The access pattern matters. The agent does not stuff its entire history into every prompt. Instead, before or during each task, the runtime queries the store with the current task's content and pulls back the few entries that look relevant. Those entries are then placed into short-term memory for this task and the model reads them as context. Long-term memory is therefore three things working together: a store, a write path that decides what is worth keeping, and a retrieval path that decides what to bring back when.

**Long-term memory** is the persistent layer of an agent's memory that survives across task boundaries. It holds user preferences, prior conversation summaries, learned facts, and reflections on past trajectories. Unlike short-term memory, which lives in the prompt and is destroyed at task end, long-term memory lives in an external store and is fetched on demand. The usual implementations cluster into three shapes. Vector databases (Pinecone, Chroma, pgvector, Weaviate, Qdrant) store text as embeddings and support similarity search, which fits fuzzy semantic retrieval like 'find conversations about Tokyo travel'. Key-value or relational stores hold structured profile data like 'user prefers tea' for exact lookup. Document or graph stores fit when entries have relationships worth traversing, like 'this user works with these colleagues'. Three operations define the memory system. Writing decides what to persist and in what form, often by summarising a finished task into a few facts. Reading retrieves the entries relevant to the current task, usually by embedding the current query and pulling top-k by similarity. Maintenance prunes, deduplicates, and ages out old or contradicted entries so the store stays useful. The critical point is that the model itself is unchanged. Long-term memory is an external augmentation, and the retrieved entries reach the model only by being placed into short-term memory for one task.

Quick reference

TL;DR

Long-term memory persists across tasks in a vector store, key-value store, or database. The agent retrieves relevant entries each task and injects them into the prompt; the model weights stay frozen.

Memory aid

Easy to grasp

Imagine a friend you only see once a month. They cannot literally remember everything you ever told them, so they keep a small notebook with the important facts: 'allergic to peanuts, has a daughter named Mia, prefers tea to coffee.' Before they meet you again, they flip to your page. Long-term memory works that way for an agent. It cannot keep every conversation alive in its head. So important facts go into a notebook (a database). Before each new task, the agent looks up the relevant page and reads it. The model's brain has not changed at all. The notebook has just grown by a few lines.

Key concepts

Long-Term Memory Vector Store Retrieval

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Define long-term memory as cross-task persistence external to the model. Name the contents and the contrast with short-term memory and weights. Describe the storage shapes (vector store, key-value, relational) and when each fits. Walk through the three operations (write, read, maintain). Close on multi-tenant scoping and aging as load-bearing production concerns.

Real-world examples

Real products, models, and research that use this idea.

Personal assistants built on the OpenAI Agents SDK or Anthropic SDK store user-fact summaries in a vector store like Pinecone or Chroma, retrieved on each new session.
Coding agents like Cursor and Devin store prior project context and reflections in a per-project memory file or vector index that the agent retrieves on each new session.
Enterprise support agents often combine pgvector for semantic memory with a relational user table for structured profile data, joined at retrieval time.
LangGraph supports checkpointed long-term memory through its store interface, with implementations for pgvector, Postgres, and Redis.
LlamaIndex offers built-in memory modules that bridge a vector index (the long-term memory) with the live chat session (the short-term memory).

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QLong-term memory and RAG both retrieve text and inject it into the prompt. What is the difference?

QWhat goes wrong when long-term memory has no aging or contradiction-handling policy?

QHow does long-term memory differ from fine-tuning the model on the user's history?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Believing long-term memory updates the model. The model weights are frozen at inference. Long-term memory is an external store the runtime queries; nothing about the model itself ever changes.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.