LLMs do not attend evenly across a long prompt. They attend more to content at the start and at the end of the context, and less to content in the middle. The pattern looks like a U when you graph retrieval accuracy against position.
The practical rule that falls out of this is simple. Put your most important context at both ends. The system prompt and role definition go at the top. Key constraints, the user query, and a final restatement of any critical rule go near the bottom, right before generation starts. The middle is where retrieved chunks, examples, or supporting material that you can afford to lose can sit.
The effect is well-documented. It persists across model sizes and across newer long-context models. Even when a model technically supports 128k or 1M tokens, the U-shape does not go away, it just gets stretched.
The empirical finding is that LLMs exhibit U-shape position bias on long contexts. They retrieve information from the start and end of the prompt reliably, and they degrade in the middle. The original paper from 2023 documented this for retrieval QA over long passages, and it has been reproduced across model families and across long-context architectures.
The practical rule is to use both ends of the prompt as high-attention real estate. Top of the prompt: system prompt, role definition, output format, the most important global constraints. Bottom of the prompt: a restatement of the key constraint, the user query itself, and any inline reminder of format. The middle is where retrieved documents, long examples, or material you do not need verbatim recall on can live.
In RAG specifically, this changes ordering. Rather than dumping retrieved chunks top to bottom by score, you place the most relevant chunk last (just before the query) and the next most relevant first. Lower-relevance chunks fill the middle. The result is measurable on grounded-QA evals: same retrieval, better answers.
Quick reference
TL;DR
LLMs show a U-shape position bias on long prompts, attending most at the head and tail; put critical context in both regions and avoid the middle.
Imagine giving a friend a giant list of instructions before they help you. They will probably remember the first few things you said and the last few things you said, and forget the stuff in the middle. Long prompts work the same way. The model pays more attention to the start of the context and the end, right before the actual question, than to anything sandwiched between them. So you put the things that really matter at both ends, and you treat the middle as the part the model is most likely to overlook.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Short prompts under a few thousand tokens The U-shape effect is weak at this length; ordering optimizations do not pay off and can over-complicate the prompt.
Single-fact lookups where one retrieval hits exactly The answer is a single hop away regardless of position; reordering is noise compared to retrieval quality.
Latency-bounded streaming where prompt construction is on the hot path Re-sorting retrieved chunks per request adds CPU and complicates prompt caching; only worth it when grounded-QA quality is the bottleneck.
Quick scan · 2–4 min
Everything important, quickly.
3 min: the U-shape empirical finding + RAG chunk reordering + long-context models do not flatten it + how to measure on your own prompts.
Real-world examples
Real products, models, and research that use this idea.
1Anthropic's RAG guidance for Claude Opus 4.7 explicitly recommends placing the most relevant retrieved document closest to the user query, citing the U-shape effect.
2Perplexity orders retrieved web snippets so the strongest source lands just before the synthesis prompt, not in score descending dump order.
3GitHub Copilot Chat reorders open-file and project-context blocks based on relevance and recency, with the highest-signal block placed adjacent to the user query.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does this change your retrieval ordering in a RAG pipeline?
Sign in or go Pro to see the approach for this follow-up.
QDoes this still apply on Gemini 3.1 Pro's 1M-token context?
Sign in or go Pro to see the approach for this follow-up.
QHow would you measure position bias on your own production prompts?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Defaulting to 'put it at the top' or 'put it at the bottom', and ignoring that the middle of a long prompt is where models are measurably worst at retrieving information.