Where in a long prompt should you place the most critical context: at the start, the middle, or the end?
LLMs show a U-shape position bias on long prompts, attending most at the head and tail; put critical context in both regions and avoid the middle.
Imagine giving a friend a giant list of instructions before they help you. They will probably remember the first few things you said and the last few things you said, and forget the stuff in the middle. Long prompts work the same way. The model pays more attention to the start of the context and the end, right before the actual question, than to anything sandwiched between them. So you put the things that matter most at both ends, and you treat the middle as the part the model is most likely to overlook.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Most prompt engineering advice treats the prompt as flat text. Put your instructions in, put your examples in, put the query in, send it off. That mental model breaks once the prompt gets long, because the model does not attend evenly across the input. There is a measurable bias, and it shapes where you should place critical content.
The finding is sometimes called Lost in the Middle, after the 2023 paper that documented it cleanly. The shape of the bias is a U: high attention at the start, high attention at the end, low attention in the middle. The effect is large enough that it changes how you should order retrieved chunks in RAG, where you place system-level constraints, and how you restate the query.
The empirical finding and why it sticks
The 2023 paper tested retrieval QA across long contexts. The authors inserted a relevant document at varying positions and asked the model to answer a question that required it. Accuracy was high when the document sat near the start, high near the end, and noticeably lower in the middle. The shape held across model sizes and across closed and open-weight families.
The tested models included an ALiBi model (MPT-30B), a position-interpolated rotary model (LongChat-13B), and the long-context versions of GPT-3.5-Turbo and Claude-1.3. A bigger context window ("we go to 128k, 1M", etc.) does not by itself erase the bias. Newer models may show a shallower U, so the exact shape has to be measured per model and task.
The practical reading is simple. Treat position in the prompt as a real input feature, not as a neutral container.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic recommends placing long documents near the top of the prompt, above the query and instructions; in its tests, queries at the end improved response quality by up to 30 percent.
- For GPT-4.1, OpenAI recommends placing instructions at both the beginning and the end of long provided context; if they appear only once, above the context works better than below.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does this change your retrieval ordering in a RAG pipeline?
Place the top-scoring chunk last (closest to query), next-best first, and let middle chunks fill the rest; cite the U-shape rationale.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Defaulting to 'put it at the top' or 'put it at the bottom', and ignoring that the middle of a long prompt is where models are measurably worst at retrieving information.
60 second bullets to scan on the way to the call.
The empirical U-shape finding and where it came from
How to order retrieved chunks for RAG
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.