How is long term memory typically implemented in an agent?

Flashcard·Easy·4.0 · 0·~30s·Asked atFiddler Ai Roblox Tesla

Attempt it

Long-term memory persists across tasks: user preferences, prior conversation summaries, learned facts, past trajectory reflections. It is usually backed by a vector database (Pinecone, Chroma, pgvector, Weaviate, Qdrant) or a key-value store; the agent retrieves relevant entries each task and injects them into the prompt. It differs from short-term memory (in-context) and from model weights (frozen at inference).

Your answer

**Long-term memory** is the part of an agent's memory that survives across tasks. Short-term memory holds what is happening right now and resets when the task ends. Long-term memory holds things the agent should remember next time: who the user is, what they prefer, what was discussed last week, what worked and what failed. It almost never lives in the model. The model's weights are frozen during inference; nothing the user says updates them. Instead, long-term memory lives in an external store that the runtime owns. The two most common shapes are a vector database (Pinecone, Chroma, pgvector, Weaviate, Qdrant) for fuzzy semantic retrieval and a key-value or relational store for exact lookups like user profiles. The access pattern matters. The agent does not stuff its entire history into every prompt. Instead, before or during each task, the runtime queries the store with the current task's content and pulls back the few entries that look relevant. Those entries are then placed into short-term memory for this task and the model reads them as context. Long-term memory is therefore three things working together: a store, a write path that decides what is worth keeping, and a retrieval path that decides what to bring back when.

**Long-term memory** is the persistent layer of an agent's memory that survives across task boundaries. It holds user preferences, prior conversation summaries, learned facts, and reflections on past trajectories. Unlike short-term memory, which lives in the prompt and is destroyed at task end, long-term memory lives in an external store and is fetched on demand. The usual implementations cluster into three shapes. Vector databases (Pinecone, Chroma, pgvector, Weaviate, Qdrant) store text as embeddings and support similarity search, which fits fuzzy semantic retrieval like 'find conversations about Tokyo travel'. Key-value or relational stores hold structured profile data like 'user prefers tea' for exact lookup. Document or graph stores fit when entries have relationships worth traversing, like 'this user works with these colleagues'. Three operations define the memory system. Writing decides what to persist and in what form, often by summarising a finished task into a few facts. Reading retrieves the entries relevant to the current task, usually by embedding the current query and pulling top-k by similarity. Maintenance prunes, deduplicates, and ages out old or contradicted entries so the store stays useful. The critical point is that the model itself is unchanged. Long-term memory is an external augmentation, and the retrieved entries reach the model only by being placed into short-term memory for one task.

TL;DR

Long-term memory persists across tasks in a vector store, key-value store, or database. The agent retrieves relevant entries each task and injects them into the prompt; the model weights stay frozen.

Memory aid

Easy to grasp

Imagine a friend you only see once a month. They cannot literally remember everything you ever told them, so they keep a small notebook with the important facts: 'allergic to peanuts, has a daughter named Mia, prefers tea to coffee.' Before they meet you again, they flip to your page. Long-term memory works that way for an agent. It cannot keep every conversation alive in its head. So important facts go into a notebook (a database). Before each new task, the agent looks up the relevant page and reads it. The model's brain has not changed at all. The notebook has just grown by a few lines.

Key concepts

Long-Term Memory Vector Store Retrieval

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Long-term memory is the persistent layer of an agent's memory: the bytes that survive after one task ends and are still available when the next task begins. Where short-term memory is the per-task scratchpad rendered into the prompt, long-term memory is the external store the runtime owns and queries selectively.

The distinction matters because of how the underlying model works. A language model at inference time is frozen. The user can talk to it for hours, and not one weight changes. Short-term memory exists only inside the prompt, and the prompt is rebuilt on each call. So every form of memory the user perceives across days, sessions, or weeks must live somewhere outside the model, in storage the runtime explicitly reads and writes.

This explanation builds up what long-term memory contains, the storage shapes used to implement it, the three operations (write, read, maintain) that govern it, and the production failure modes a serious design has to handle. By the end you should be able to look at any 'the agent remembers me' feature and decompose it into these pieces.

What long-term memory contains and where it lives

Three categories of content account for most production long-term memory.

The first is user profile data: stable facts about the user (name, language, role, preferences, allergies, dietary restrictions, account identifiers, locale). These tend to be small, structured, and updated rarely.

The second is interaction history summaries. A finished task is summarised down to a compact set of facts (the user planned a Tokyo trip arriving 9am Haneda, allergic to peanuts, asked about ramen restaurants) and the summary is persisted. Raw transcripts are usually too large to retrieve efficiently, so the summary is the primary artifact, with the trace remaining available in observability.

The third is learned heuristics or reflections. After repeated failure modes, the agent or a meta-agent writes down lessons like 'when the user says urgent, escalate to overnight shipping' or 'this customer's payment method fails on weekends'. These are the cross-task equivalent of Reflexion-style critiques and are explicitly retrieved when the agent decides what to do next.

Storage choices follow content shape. Free text and summaries go into a vector store for similarity retrieval; common 2026 choices are Pinecone, Chroma, pgvector (used heavily by teams already on Postgres), Weaviate, and Qdrant. Structured profile data goes into a relational or key-value store with exact lookup. Relationships between entities (this user belongs to this team, used these products) fit a graph or document store. Production systems often combine two or three: a relational user table joined with a pgvector table of summaries, queried together at retrieval time.

Three operations: write, read, maintain

How long-term memory reaches the model

Production failure modes and how to design against them

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Personal assistants built on the OpenAI Agents SDK or Anthropic SDK store user-fact summaries in a vector store like Pinecone or Chroma, retrieved on each new session.
Coding agents like Cursor and Devin store prior project context and reflections in a per-project memory file or vector index that the agent retrieves on each new session.

What an interviewer would ask next. Try answering before peeking at the approach.

QLong-term memory and RAG both retrieve text and inject it into the prompt. What is the difference?

RAG retrieves from a static or shared knowledge base (docs, web pages, manuals) to answer questions. Long-term memory retrieves from a per-user write-back store of past interactions. The mechanics overlap (vector store + retrieval), but the source of the data and the write path differ sharply.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Believing long-term memory updates the model. The model weights are frozen at inference. Long-term memory is an external store the runtime queries; nothing about the model itself ever changes.

60 second bullets to scan on the way to the call.

Define long-term memory as cross-task persistence external to the model.
Name the contents: user preferences, prior summaries, learned facts, trajectory reflections.

Primary sources. Browse if you want the original framing.