Predict the retrieval trace: how a multi-hop RAG agent answers a chained question that single-shot RAG would fail on.
A multi-hop RAG agent is given the question: 'What was the revenue impact in fiscal year 2025 of the cloud product launched by the CEO who replaced the founder of Stripe?' The agent runs an iterative loop where each step issues one retrieval query, reads the chunks, and decides the next query (Self-Ask / IRCoT style). Assume the corpus contains the relevant Wikipedia and SEC-filing chunks. Write the sequence of retrieval queries the agent issues, in order, ending when it has enough to answer. For each query, in one phrase say what fact it surfaces.
The question chains three facts: who replaced the founder, what that person launched, and the revenue impact. Each hop depends on the previous answer, so the agent issues three sequential queries.
Imagine someone asks you who painted the ceiling of the chapel where Pope Julius II is buried. You cannot answer that in one library trip. First you find out where Pope Julius II is buried, then you find out who painted the ceiling of that chapel. Two trips, and the second trip needs the answer from the first. Multi-hop RAG works the same way. Each step asks one sub-question, looks at the answer, and uses it to write the next sub-question. A single-shot search would not work because no single book has the full chain in one place. The agent strings together separate facts by doing one careful lookup at a time.
Detailed answer & concept explanation~7 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
8 min: walk the three hops, explain the dependency between them, note the latency cost, and contrast with single-shot RAG.
# Sketch of a multi-hop retrieve and reason loop, IRCoT-style.
from anthropic import Anthropic
client = Anthropic()
async def multihop_answer(question: str, max_hops: int = 5) -> tuple[str, list[dict]]:
facts: list[dict] = [] # accumulated (sub_q, chunk, extracted_fact)
for hop in range(max_hops):
# 1. Decide the next sub-question given what we already know.
plan = await client.messages.create(
model="claude-opus-4-7",
messages=[{"role": "user", "content": plan_prompt(question, facts)}],
)
sub_q = parse_subquestion(plan)
if sub_q is None:
break # planner says we have enough
# 2. Retrieve for this sub-question.
chunk = await retrieve_top_chunk(sub_q)
# 3. Extract the fact relevant to the chain.
extracted = await extract_fact(sub_q, chunk)
facts.append({"sub_q": sub_q, "chunk": chunk, "fact": extracted})
# 4. Synthesize final answer with citations.
return await synthesize(question, facts)| Aspect | Single-shot RAG | Multi-hop RAG |
|---|---|---|
| Queries per question | 1 | k (typically 2-5) |
| Latency | ~1.5 s P95 | ~k × 1.5 s P95 |
| LLM calls | 1 | k+1 (one per hop + synthesis) |
| Cost multiplier | 1x | 2-5x |
| Strength | single-fact questions | chained / comparative questions |
| Failure mode | misses chain facts | error propagates across hops |
Real products, models, and research that use this idea.
- LangChain Self-Ask with Search agent runs this exact retrieve and reason loop, using a follow-up question marker to drive each hop.
- DSPy's IRCoT module implements the IRCoT pattern, interleaving chain-of-thought reasoning with retrieval at each step.
- Anthropic's Claude 4.7 tool-use loop is commonly wired to a retrieval tool for multi-hop, with the model deciding when to stop.
- Perplexity's Pro Search runs an iterative search loop where each follow-up query is conditioned on what the previous turn surfaced.
- Adaptive RAG (Jeong et al. 2024) gates multi-hop behind a complexity classifier so simple queries skip the extra cost.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does the agent know when to stop the loop?
QWhat happens if hop 1 retrieves the wrong CEO name? How does the system recover?
QCompare multi-hop iterative retrieval with multi-query parallel retrieval (HyDE / multi-query fusion).
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Issuing one query against the raw chained question. A single search cannot surface three distinct facts at once if no chunk contains the full chain.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
- Press, Ofir et al., Measuring and Narrowing the Compositionality Gap in Language Models (Self-Ask)
- Trivedi et al., Interleaving Retrieval with Chain-of-Thought Reasoning (IRCoT)
- Jeong et al., Adaptive-RAG: Learning to Adapt Retrieval-Augmented LLMs through Question Complexity
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop QA
Same topic, related formats. Practice these next.