Predict the retrieval trace: how a multi-hop RAG agent answers a chained question that single-shot RAG would fail on.

Attempt it

A multi-hop RAG agent is given the question: 'What was the revenue impact in fiscal year 2025 of the cloud product launched by the CEO who replaced the founder of Stripe?' The agent runs an iterative loop where each step issues one retrieval query, reads the chunks, and decides the next query (Self-Ask / IRCoT style). Assume the corpus contains the relevant Wikipedia and SEC-filing chunks. Write the sequence of retrieval queries the agent issues, in order, ending when it has enough to answer. For each query, in one phrase say what fact it surfaces.

Quick reference

TL;DR

The question chains three facts: who replaced the founder, what that person launched, and the revenue impact. Each hop depends on the previous answer, so the agent issues three sequential queries.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine someone asks you who painted the ceiling of the chapel where Pope Julius II is buried. You cannot answer that in one library trip. First you find out where Pope Julius II is buried, then you find out who painted the ceiling of that chapel. Two trips, and the second trip needs the answer from the first. Multi-hop RAG works the same way. Each step asks one sub-question, looks at the answer, and uses it to write the next sub-question. A single-shot search would not work because no single book has the full chain in one place. The agent strings together separate facts by doing one careful lookup at a time.

Key concepts

RAG system design Embeddings Vector databases

Quick scan · 2–4 min

Everything important, quickly.

8 min: walk the three hops, explain the dependency between them, note the latency cost, and contrast with single-shot RAG.

Code example

python

# Sketch of a multi-hop retrieve and reason loop, IRCoT-style.
from anthropic import Anthropic
client = Anthropic()

async def multihop_answer(question: str, max_hops: int = 5) -> tuple[str, list[dict]]:
    facts: list[dict] = []  # accumulated (sub_q, chunk, extracted_fact)
    for hop in range(max_hops):
        # 1. Decide the next sub-question given what we already know.
        plan = await client.messages.create(
            model="claude-opus-4-7",
            messages=[{"role": "user", "content": plan_prompt(question, facts)}],
        )
        sub_q = parse_subquestion(plan)
        if sub_q is None:
            break  # planner says we have enough

        # 2. Retrieve for this sub-question.
        chunk = await retrieve_top_chunk(sub_q)

        # 3. Extract the fact relevant to the chain.
        extracted = await extract_fact(sub_q, chunk)
        facts.append({"sub_q": sub_q, "chunk": chunk, "fact": extracted})

    # 4. Synthesize final answer with citations.
    return await synthesize(question, facts)

Side-by-side

Aspect	Single-shot RAG	Multi-hop RAG
Queries per question	1	k (typically 2-5)
Latency	~1.5 s P95	~k × 1.5 s P95
LLM calls	1	k+1 (one per hop + synthesis)
Cost multiplier	1x	2-5x
Strength	single-fact questions	chained / comparative questions
Failure mode	misses chain facts	error propagates across hops

Real-world examples

Real products, models, and research that use this idea.

LangChain Self-Ask with Search agent runs this exact retrieve and reason loop, using a follow-up question marker to drive each hop.
DSPy's IRCoT module implements the IRCoT pattern, interleaving chain-of-thought reasoning with retrieval at each step.
Anthropic's Claude 4.7 tool-use loop is commonly wired to a retrieval tool for multi-hop, with the model deciding when to stop.
Perplexity's Pro Search runs an iterative search loop where each follow-up query is conditioned on what the previous turn surfaced.
Adaptive RAG (Jeong et al. 2024) gates multi-hop behind a complexity classifier so simple queries skip the extra cost.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does the agent know when to stop the loop?

Sign in or go Pro to see the approach for this follow-up.

QWhat happens if hop 1 retrieves the wrong CEO name? How does the system recover?

Sign in or go Pro to see the approach for this follow-up.

QCompare multi-hop iterative retrieval with multi-query parallel retrieval (HyDE / multi-query fusion).

Sign in or go Pro to see the approach for this follow-up.

QHow would you train or fine-tune a model specifically for the multi-hop sub-question generation step?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.