Lost in the Middle
Models attend best to the start and end of long contexts and lose info in the middle.
A documented LLM failure mode where models pay disproportionate attention to information at the beginning and end of long contexts while neglecting the middle. Has direct implications for RAG ordering and long-context prompting.
In practice
Why RAG result ordering matters and why long-context windows don't automatically help. Standard senior-level gotcha question.
Related topics
Questions that mention this term
- Your team is tuning top-k for the retriever in a production RAG system. What is the strongest reason NOT to just push top-k to its highest possible value?
- Order the stages of a retrieval and assembly pipeline that exploits the lost-in-the-middle curve
- Which curve shape does the lost-in-the-middle effect produce?
- What role does the LLM's context window play in a RAG pipeline?
- Select the practical mitigations for the lost-in-the-middle effect that move the needle
- What is context pollution in a long running agent and what mitigation strategies are available?
Related terms
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.
BM25
Classical TF-IDF-style lexical scoring: a surprisingly strong baseline, still the first stage of many hybrid RAG pipelines.
Chunking
Cutting documents into retrievable pieces before embedding them.
Context Engineering
Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.
Context Rot
Long-running chats degrade: early instructions get forgotten, and tool calls become less reliable.
Context Window
The max number of tokens a model can attend to at once.