Why does restating the user question right before the answer position help with long contexts?

Flashcard·Medium·4.0 · 0·~30s·Asked atAnthropic OpenAI Cohere

Attempt it

Repeating the question right before the answer position anchors the model's attention on the task at the recency end of the context, where attention is strongest. Without the restate, the question lives at the top, separated from the answer by thousands of tokens of retrieved evidence, and the lost-in-the-middle effect makes the model effectively forget what was asked while it processes the evidence. The failure mode prevented is on topic but off question answers: the model summarizes the documents rather than answering the original question.

Your answer

In a typical RAG prompt, the user's question sits at the top, then comes a long block of retrieved chunks, then the model is expected to answer. If the retrieved block is a few thousand tokens, the model finishes reading the evidence and the original question is now far above the answer position, separated by the entire retrieval block. Language models have **strong recency bias**. The bytes immediately before the answer position get the most attention. When the only thing in that strong-attention zone is retrieved evidence, the model's natural pull is to summarize or paraphrase the evidence, not to answer the original question. The restate fix is simple: after the retrieved chunks and right before the model's answer position, repeat the user's question. Now the strongest-attention zone contains the actual task. The model still has the evidence available, it does not delete the chunks, but the task description is freshest in mind when generating the answer. The failure mode this prevents is **on topic but off question** answers. The model writes something that is clearly about the right subject matter, often well-written, but never addresses the specific thing the user asked. The restate fixes this almost every time.

Restating the user's question right before the answer position is one of the cleanest, cheapest, and most reliable long-context tricks in 2026 RAG. The mechanism is recency exploitation. Autoregressive models predict the next token based most strongly on tokens immediately before the prediction point. In a typical RAG prompt, system prompt, then question, then 5,000 tokens of retrieved chunks, then answer, the question is thousands of tokens upstream of the answer position. Recency attention is fully captured by the evidence block. Without the restate, the model's strongest local signal is 'summarize or interpret this evidence.' Without an explicit reminder of *what the user asked*, the model defaults to producing a coherent gloss of the retrieved content. The output looks fine on the surface, it is on-topic, well-written, and grounded in the chunks, but it never answers the specific question. The failure mode has a name in production circles: **on topic but off question**. The user asks 'what is the cheapest flight from JFK to SFO under 6 hours?' The retriever pulls 12 flight listings. The model responds with 'Here are the available flights from JFK to SFO, ranging in price and duration...', accurate, grounded, useless. The restate places the question (often with a verb-form imperative, 'Answer the user's question:') immediately before the answer position. The recency slot now holds the task. The model still has the evidence above, but it generates with the question freshest in mind. Latency cost: roughly 20-50 tokens. Quality gain: well-documented in production evals; many teams report 5-15 percent accuracy improvement on long-context QA tasks for this single change.

Quick reference

TL;DR

Restating the question right before the answer position anchors the task in the recency slot, where attention is strongest, and prevents the model from drifting into 'summarize the evidence' mode.

Memory aid

Easy to grasp

Imagine asking a friend a specific question, then handing them a 200-page report and asking them to answer using the report. By the time they have read the report, the specific question has faded; they will tend to summarize the report instead of answering you. Now imagine you slip the question onto a sticky note and attach it to the last page. Right when they finish reading and pick up the pen, the question is the freshest thing they saw. They answer the question, not the report. The restated question is that sticky note. It works for the same reason: it puts the goal where attention is strongest, right before the answer.

Key concepts

Context Engineering

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 to 5 min: explain the recency mechanism, walk through the on topic but off question failure mode, and demonstrate the restate pattern with placement before vs after the chunks.

Real-world examples

Real products, models, and research that use this idea.

Anthropic's prompt engineering guide for Claude explicitly recommends placing the user query both before and after the document content for long-document QA.
OpenAI's RAG documentation suggests structuring the prompt as instructions, then context, then question (the question last, by design).
LangChain's `create_stuff_documents_chain` defaults to placing the question at the bottom of the assembled prompt below the documents.
LlamaIndex's response synthesizer templates put `Query: {query}` at the end of the prompt, after `Context: {context}`.
Cohere's command-r model card calls out the recency-question pattern as part of its retrieval-aware prompt template.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat happens if you restate the question multiple times throughout the prompt?

QHow does this pattern interact with structured-output schemas?

QCan you measure the lift from question restate on your own pipeline?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Placing the user question only at the top of a long RAG prompt, then watching the model summarize the retrieved chunks instead of answering the actual question.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.