Context Rot
Also known as: Conversation degradation
Long-running chats degrade: early instructions get forgotten, and tool calls become less reliable.
Degradation of LLM behavior as the conversation history grows long: earlier turns get less attention, instructions get re-interpreted, and tool calls become less reliable. Mitigated by summarization, context engineering, and periodic resets.
In practice
Single most underappreciated failure mode in production agents. Senior LLM app interviews probe summarization and context refresh patterns.
How it compares
Lost-in-the-middle is a positional attention bias; context rot is a longitudinal degradation as history grows.
Related topics
Related terms
Agent Loop
The runtime: LLM call → tool call → observation → LLM call → … until a final answer or step cap.
AI Agents
LLMs that loop: plan → call tools → observe results → repeat until done.
Chain-of-Thought (CoT)
Ask the model to think step by step before answering. It boosts accuracy on reasoning tasks.
Chunking
Cutting documents into retrievable pieces before embedding them.
Context Engineering
Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.
Fixed-size Chunking
Slice documents every N tokens with a small overlap. Fast and format-agnostic, but cuts through headings and mid-topic.