Compare eager memory injection with lazy tool-callable recall

Attempt it

Eager injection retrieves the most relevant memories at turn start and prepends them to context. It is simple, deterministic, and adds zero latency once retrieved, but it pays the token cost on every turn whether the model needed those memories or not, and it can crowd the budget on memory-heavy users. Lazy recall exposes a search_memory tool the model invokes only when it determines a memory is needed; it pays nothing on turns that do not need recall, but adds a tool-call round-trip when it does. Eager wins for conversational assistants where memory is usually relevant. Lazy wins for task-focused agents where most turns are about the task and only occasional ones need history.

Your answer

**Eager memory injection** retrieves the most relevant stored memories at the start of every turn and inserts them into the prompt. The retrieval happens before the model sees the user's new message, typically via a vector search over the memory store using the new message as the query. The retrieved memories are prepended to context as a memory block. The upside is simplicity and zero in-loop latency. The model always has access to potentially relevant memory without needing to ask for it. The trajectory is deterministic and easy to reason about. The downside is **fixed token cost per turn**. Every turn pays for the memory block, even turns where the model would not have needed any memory. On long sessions with rich memory stores, this is expensive and can crowd out room for the actual task. **Lazy tool-callable recall** does the opposite. Memory lives behind a `search_memory(query)` tool that the model invokes only when it judges that recall is needed. Turns that are about the task at hand pay nothing for memory. Turns where the model needs to remember something pay one tool-call round-trip, the model emits a tool call, the system runs the search, the result comes back, and the model continues. The upside is **on-demand cost**, only pay when memory is used. The downside is the extra latency on turns that do use it, plus the model has to be smart enough to know when to call. The right pattern depends on the workload. Conversational assistants where the user's history is usually relevant favor eager. Task-focused agents where most turns are about the immediate task favor lazy. Many production systems use a hybrid.

**Eager memory injection** and **lazy tool-callable recall** are the two canonical patterns for surfacing long-term memory in 2026 agent and assistant designs, and they optimize for opposite ends of the latency versus tokens tradeoff. **Eager injection** is the always-on pattern. At the start of every turn, a retriever runs a vector search over the memory store using the new user message as the query and returns the top-k memories. Those memories are formatted into a compact block (often a few hundred tokens) and prepended to context before the model is invoked. The model sees relevant memory without having to ask for it. The properties of eager: **simple architecture** (one retrieval call per turn, deterministic), **zero in-loop latency** (the retrieval happens before model invocation and adds at most tens of milliseconds), **predictable trajectory** (no tool-call branch in the model's output). The cost is a **fixed token tax per turn**, the memory block occupies prompt space whether the model uses it or not. On memory-heavy users this can be 1-3k tokens per turn, every turn. **Lazy recall** is the on-demand pattern. Memory lives behind a `search_memory(query)` tool. The model decides per turn whether to call it. Turns that are about the task at hand do not invoke memory and pay zero token cost. Turns that need history emit a tool call, the system runs the search, results come back, the model continues. The properties of lazy: **token-cheap on average** (only pay when recall is needed), **on-demand latency** (a tool-call round-trip, typically 300-1000ms of added wall-clock per call), **less predictable trajectory** (the model may or may not call the tool; instruction quality matters). **Choosing between them is a workload-shape question.** Eager wins when memory is needed on most turns: conversational assistants, support chat with user history, personal AI.

Quick reference

TL;DR

Eager injection prepends retrieved memories every turn, simple and low-latency but pays token cost always; lazy recall calls a search tool only when needed, token-cheap but adds a tool round-trip when used.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine an assistant who either keeps your notes spread on the desk in front of them all the time, or keeps them filed away and walks to the cabinet only when they need to look something up. The first way means they never miss a relevant note, but the desk is crowded and you pay for the space. The second way keeps the desk clean and cheap, but every cabinet trip is a small delay. Neither is better in general. If you ask about your notes constantly, leaving them on the desk is the right call. If you mostly do other work and only sometimes need them, the cabinet wins.

Key concepts

Context Engineering

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

Describe eager injection (retrieve at turn start, prepend to context) and its fixed token cost. Describe lazy recall (search tool invoked by model judgment) and its on-demand round-trip cost. Frame the choice as a workload-shape question driven by how often memory is actually needed. End with the hybrid pattern that production systems converge on: a small eager persistent block plus lazy episodic and semantic recall.

Real-world examples

Real products, models, and research that use this idea.

Mem0's per-turn fact injection is a textbook eager pattern, with extracted facts retrieved before each model call
Letta's memory-block edit tools implement the lazy pattern, with the model deciding when to read or write archival memory
ChatGPT's memory feature uses a hybrid: a small always-on persistent block plus a larger archive accessed on demand
LangGraph applications often combine eager state slots (the user profile, goals) with lazy tools for episodic recall
Anthropic's Claude memory tool in 2026 supports both injection and tool-driven access patterns

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you evaluate which pattern is right for a new product without committing in advance?

Sign in or go Pro to see the approach for this follow-up.

QWhat goes wrong when you stack eager injection on top of a long retrieved-document context?

Sign in or go Pro to see the approach for this follow-up.

QWhy does lazy recall benefit from a careful tool description more than eager does?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Picking the eager pattern by default because it is simpler, then watching the prompt budget bloat with memories the model never used on most turns.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.