Explain why treating a LangChain `Memory` object (or its `RunnableWithMessageHistory` successor with an in-memory store) as durable conversation storage is a production footgun. What is the framework actually responsible for, and what is on you?
Free · 2 AI evals / day
**LangChain's 'memory' is a confusing word.** The framework uses it to mean 'how the conversation history is injected into the next prompt,' not 'where the conversation is stored permanently.' Those are two different problems, and conflating them is the footgun.
**What the framework owns: window management.** A memory class (or in modern LangChain a `RunnableWithMessageHistory`) handles how prior turns get formatted into the prompt, how the history is trimmed when the context window fills up, whether old turns are summarised or dropped, and how the next user input is appended.
**What you own: persistence.** Where the message list actually lives. If you configure `InMemoryChatMessageHistory` (the default for tutorials), the list lives in a Python dict inside the worker process. When the process restarts (deploy, crash, container recycle), the dict is gone and so is the conversation. When the next request lands on a different worker (load balancer, autoscale), the dict on that worker is empty and the conversation appears to start over.
The fix is to wire the history to a persistent backend: `RedisChatMessageHistory`, `PostgresChatMessageHistory`, `DynamoDBChatMessageHistory`, or your own implementation against your existing store. The framework's job ends at 'use this history when threading the prompt'; the persistence is your responsibility.
**Two responsibilities, one easily-confused word.**
LangChain's 'memory' refers to window management: the policy for how prior turns are injected into the next prompt, how old turns are summarised or dropped when the context window fills, and how new messages are appended. The framework owns this; it provides classes like `ConversationBufferMemory`, `ConversationSummaryMemory`, `EntityMemory`, and in modern LangChain the `RunnableWithMessageHistory` wrapper that handles the same job at the Runnable layer.
Persistence (where the message list actually lives across requests, processes, and restarts) is a separate responsibility, and the framework does not own it. The default `InMemoryChatMessageHistory` is a Python dict in the worker process; it works for notebooks and tutorials and fails the moment the application is restarted, scaled, or load-balanced across multiple workers.
**The fix.** Wire the history to a real backend. `RedisChatMessageHistory` is the common pick for chat applications (low-latency, distributed). `PostgresChatMessageHistory` is the durable-relational pick. `DynamoDBChatMessageHistory` for AWS-native. You can also implement the `BaseChatMessageHistory` interface against any store you already run.
**The analogous trap appears in LangGraph.** Skip the checkpointer and the graph runs in memory; the moment the process restarts, the conversation state evaporates. Same division of labour: the framework provides state-machine semantics, you provide persistence.
**The deeper principle.** Frameworks own the orchestration of stateful workflows; they do not own the durability of that state. Treating them as databases is the architectural mismatch that produces this footgun across multiple framework surfaces.
Quick reference
TL;DR
LangChain memory is window management (what goes in the prompt), not durable storage; persistence is your job via Redis, Postgres, or another real backend.
Imagine an assistant who scribbles your conversation onto a sticky note so they can remember what you said earlier in the same chat. Every time you restart the assistant or talk to a different one, the sticky note vanishes. LangChain memory is the sticky note: it helps the model remember within one chat, but it does not save the chat in a way that survives a coffee break, let alone a server restart. If you want the chat to survive, you have to give the assistant a real notebook (a database) to write in.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Notebooks, tutorials, and one-shot scripts where conversations do not need to survive the process In-memory history is perfectly fine here; the framework's default is calibrated for this case. The footgun appears only when the same code ships to production unchanged.
Stateless single-turn requests where no history is needed Skip memory entirely. Pass each request as a fresh prompt; the simpler architecture has no persistence concern.
Workflows where the conversation history is a side effect already stored elsewhere (the user's CRM, the ticket system, the chat log) Loading from that store on every request may be simpler than wiring a separate persistent backend; the existing store is your persistence layer.
Quick scan · 2–4 min
Everything important, quickly.
5 min: window management vs persistence as separate responsibilities + RunnableWithMessageHistory mechanics + production failure modes + persistent backend options + analogous LangGraph checkpointer trap + summarisation for unbounded growth.
Real-world examples
Real products, models, and research that use this idea.
1Klarna's customer-support chat uses RedisChatMessageHistory so conversation state survives the rolling deploy of their assistant fleet.
2Linear's AI features use Postgres-backed chat history because the conversation is part of the audit trail for the issue.
3Replit Agent uses durable backend storage for conversation state so a tab close does not lose the agent's working context.
4Notion AI's chat surface persists history in Notion's own data store, exposed via a custom BaseChatMessageHistory implementation.
5Many tutorial code samples ship with InMemoryChatMessageHistory and a comment saying 'replace with a real backend in production', representative of the default-trap.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWalk through implementing a BaseChatMessageHistory against your own Postgres schema.
Sign in or go Pro to see the approach for this follow-up.
QHow do you handle message-history growth in long-running conversations?
Sign in or go Pro to see the approach for this follow-up.
QHow does this compare to LangGraph's checkpointer story?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Assuming `ConversationBufferMemory` or `InMemoryChatMessageHistory` will keep a conversation alive in production. They live in process memory and die on restart, crash, or autoscale event.