How is a RAG vector store used as an external long term memory layer inside an agent?

An LLM forgets everything between calls. Its only working memory is the context window, and once a conversation ends, that memory is gone. **RAG as memory** is a way to give an agent a durable store of past facts and interactions that it can pull from later. The idea is simple. You take past interactions, facts, or documents, turn each one into an embedding (a list of numbers that captures its meaning), and save them in a vector store. Later, when the agent needs to remember something, it writes a query, the store finds the closest matching entries, and those entries come back as text. That returned text is then placed into the agent's context as an observation, exactly like the result of any other tool call. The model reads it on the next turn and uses it to answer. The model's own weights never change. The memory lives entirely outside the model. The key thing to grasp is that the agent does not really store memories inside its brain. It looks them up fresh each time, reads the few that matter, and forgets them again once the turn is over.

**RAG as memory** treats a vector store as an agent's long term memory layer. Past facts and interactions are embedded and saved. The agent later retrieves them on demand and reads them in context. The write path runs first. After a step or a conversation, you embed the new fact and store the vector alongside its text. The read path runs during the loop. When the agent needs information it lacks, it issues a retrieval tool call with a query drawn from its current reasoning. The store runs a similarity search and returns the top matches. The ranking is a cosine score: $$s(q, d) = \frac{q \cdot d}{\lVert q \rVert \, \lVert d \rVert}$$ The runtime injects those chunks as an Observation. They become ordinary context the model reads on its next turn. Crucially, the model weights never change. Memory is in context injection, not fine-tuning. The trigger pattern is usually demand driven. The agent retrieves only when a need signal fires, such as low confidence. This keeps cost down versus retrieving on every step. The contrast is fine-tuning, which bakes facts into weights but cannot be updated cheaply or audited per fact.