Lost in the Middle
Models attend best to the start and end of long contexts and lose info in the middle.
A documented LLM failure mode where models pay disproportionate attention to information at the beginning and end of long contexts while neglecting the middle. Has direct implications for RAG ordering and long-context prompting.
In practice
Why RAG result ordering matters and why long-context windows don't automatically help. Standard senior-level gotcha question.
Related topics
Questions that mention this term
- Pick the curve shape that the lost-in-the-middle effect produces
- Select the practical mitigations for the lost-in-the-middle effect that actually move the needle
- What is context pollution in a long running agent and what mitigation strategies are available?
- What causes context pollution in a long running agent?
- Why cap tool output size at the tool boundary instead of letting the model deal with it?
- Which claim about retrieving more chunks per query is correct?
Related terms
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.
BM25
Classical TF-IDF-style lexical scoring: a surprisingly strong baseline, still the first stage of many hybrid RAG pipelines.
Chunking
Cutting documents into retrievable pieces before embedding them.
Context Engineering
Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.
Context Rot
Long-running chats degrade: early instructions get forgotten, and tool calls become less reliable.
Context Window
The max number of tokens a model can attend to at once.