Three options sit on the table when you design a multi-turn chat: keep everything, keep nothing past a reset point, or compress old turns into a summary. Each one trades off differently.
Keeping everything is the simplest. It works well when the next turn genuinely depends on earlier ones, like a code edit thread or a step by step plan. The cost is tokens, which grow every turn, and attention, which gets diluted across noise.
Resetting clears the slate. It works best when the user changes topic, because old turns become noise that competes with the new question. Summarizing compresses old turns into a short paragraph and keeps that paragraph in context, which is what you reach for when the chat has run for a long time but the gist still matters.
The practical rule: pick based on what the next turn actually needs, not on turn count.
Multi-turn chat needs an explicit strategy for history, not a default. The three real strategies are reset, carry full, and summarize, and each one is right for a different regime.
Reset when the topic shifts substantively. Old turns are now noise: they compete for attention and burn tokens without contributing to the answer. A good heuristic is to detect topic change with a lightweight classifier or a salience score and flush before the next prompt.
Carry full when each turn genuinely depends on prior ones. Code editing sessions, multi-step reasoning over a shared artifact, and back and forth debugging all fall here. The cost is real but the value is real too.
Summarize when history is long but the gist matters. Long support sessions, multi-day project threads, ongoing onboarding flows. Practical thresholds people use: more than around 20 turns, more than around 8k tokens, or whenever the conversation is about to bump the context window. The summary itself is a prompt-engineering artifact, with its own template and quality bar.
Quick reference
TL;DR
Reset on topic shift, carry full when turns depend on each other, summarize when history is long but the gist matters; pick by coupling and budget, not turn count.
Imagine taking notes during a long meeting. Sometimes the topic changes completely and your old notes are just clutter, so you flip to a fresh page. Sometimes every sentence builds on the last and you keep the full transcript in front of you. Sometimes the meeting has been going for hours and you cannot read it all, so you write a one-paragraph summary and use that instead. Multi-turn LLM chats work the same way. The right choice depends on whether the new turn really needs everything that came before, only the gist, or none of it.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Short, single-topic conversations under a handful of turns History strategy is overhead; just carry full. The token cost is trivial and there is no shift to detect.
Anything safety-critical where a missed turn breaks correctness Summarization is lossy by design; for legal or clinical workflows, carry full and let the budget take the hit.
Stateless tool-use agents where each call is independent No history strategy applies. Pass only the per-call inputs; carrying state confuses the agent and burns tokens.
Quick scan · 2–4 min
Everything important, quickly.
3 min: three strategies + topic-shift vs coupling triggers + summarization template + hybrid production pattern + cache interaction.
Real-world examples
Real products, models, and research that use this idea.
1OpenAI Assistants exposes a thread abstraction with automatic truncation and summarization helpers, so teams pick a strategy without hand-rolling the loop.
2Anthropic Claude Projects keeps a verbatim recent window plus a project-level summary that survives across sessions, mirroring the hybrid pattern.
3Cursor and GitHub Copilot Chat default to a verbatim recent-turn window for code edits because coupling is high; longer threads get summarized into a compact preface.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you detect a topic shift reliably?
Sign in or go Pro to see the approach for this follow-up.
QHow does prompt caching interact with history strategy?
Sign in or go Pro to see the approach for this follow-up.
QWhat does a good summary template look like?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Carrying full history every turn because losing context feels dangerous, while ignoring the token cost, the position-bias dilution, and the noise that irrelevant turns add when the topic has shifted.