Reset on topic shift, carry full when turns depend on each other, summarize when history is long but the gist matters; pick by coupling and budget, not turn count.
Imagine taking notes during a long meeting. Sometimes the topic changes completely and your old notes are just clutter, so you flip to a fresh page. Sometimes every sentence builds on the last and you keep the full transcript in front of you. Sometimes the meeting has been going for hours and you cannot read it all, so you write a one-paragraph summary and use that instead. Multi-turn LLM chats work the same way. The right choice depends on whether the new turn really needs everything that came before, only the gist, or none of it.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Multi-turn chat looks simple from the outside. The user sends a message, the model responds, the next turn arrives. The complexity hides in what you put in the prompt every turn after the first. That is the history strategy, and it is one of the most consequential decisions in any LLM application.
The three strategies (reset, carry full, summarize) are not preferences. They are different answers to a budget allocation problem. You have a context window, a position-bias curve, a per token cost, and a latency budget. History strategy is how you spend those resources across the conversation.
Reset, the cheapest signal you can send
Reset means flushing prior turns before the next prompt. The right trigger is topic shift, not turn count. When the user pivots from one task to another, old turns become noise. They compete for attention against the new context and burn tokens for no benefit.
Detecting the shift is itself an engineering choice. Embedding similarity between consecutive turns is a cheap signal: a sharp drop usually means a topic change. A small classifier or even a keyword heuristic works for narrower domains. Some products expose an explicit "New chat" button and lean on the user to make the call.
The failure mode here is resetting too aggressively. If you flush every few turns, coupled conversations like debugging or stepwise reasoning lose context and the user has to repeat themselves. Tune the threshold. Do not pick a fixed cadence.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- OpenAI's Responses API can keep conversation state on the server through previous_response_id or a Conversations object, but every earlier input token in the chain is still billed as input, so the history strategy still drives cost.
- LangChain's legacy ConversationSummaryBufferMemory (deprecated, now in the langchain-classic package) keeps recent turns verbatim and summarizes older turns once a token limit is passed, which is the hybrid pattern in library form. LangChain 1.x moves this job to a summarization middleware.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you detect a topic shift reliably?
Combine an embedding-similarity drop between consecutive turns with a small classifier or keyword heuristic; flush below a tuned threshold.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Carrying full history every turn because losing context feels dangerous, while ignoring the token cost, the position-bias dilution, and the noise that irrelevant turns add when the topic has shifted.
60 second bullets to scan on the way to the call.
When to reset vs carry vs summarize
Topic shift as the trigger, not turn count
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.