Agentic RAG puts retrieval inside the agent loop as a tool the model can call multiple times. Single-shot RAG retrieves once before generation. Agentic RAG handles multi-hop and missing-context cases at extra cost.
Imagine answering a question with a stack of books. One way: someone hands you five pages they think are relevant, and you write your answer using only those pages. That is single-shot RAG. Another way: you can ask for any page at any time, read it, decide if you need more, ask for another, and so on, until you feel you have enough. That is agentic RAG. The first is faster because you only look once. The second is better when the answer needs you to chase down related facts, but it costs more because you keep looking. Most modern AI products like Perplexity research mode and ChatGPT browsing use agentic RAG for hard questions and fall back to single-shot for simple ones.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Agentic RAG is the generalisation of single-shot RAG where retrieval becomes a tool inside an agent loop instead of a fixed pre-processing step. The model decides when to query, what to query, whether the result is sufficient, and whether to follow up with another retrieval.
The pattern emerged naturally once ReAct became standard. If the model can call tools mid-task, and retrieval is just another tool, then there is no reason to hard-code retrieval as a one-time step before generation. Letting the model decide retrieval at runtime is more flexible, handles harder questions, and costs more.
This explanation defines both patterns precisely, explains the problem classes where agentic RAG wins, lays out the cost reality, and covers the routing strategy production systems use to balance the two.
Single-shot RAG: the baseline
Single-shot RAG is the original RAG pipeline (Lewis et al. 2020). The flow has three fixed steps.
Embed. The user's query is converted into a dense vector using an embedding model. Modern systems often use multiple representations: a dense embedding plus a sparse representation like BM25 for hybrid retrieval.
Retrieve. The vector database returns the top-K chunks most similar to the query embedding. K is typically 5 to 20. Reranking (using a cross-encoder model to re-score the top candidates) often follows.
Generate. The retrieved chunks are prepended to the model's prompt as context, and the model generates the answer conditioned on them.
This is the workhorse RAG pattern. It is fast, cheap, and easy to debug. The retrieval happens exactly once, with no model decision involved beyond what query the user provided. The pipeline is deterministic given the embedding model and the corpus state.
Single-shot RAG is the right choice for simple lookup questions: 'What is the return policy?', 'Who founded the company?', 'What does the configuration parameter X do?'. The answer is usually in one chunk and one retrieval suffices.
The failure modes are predictable. Single-shot misses multi-hop questions where the answer spans linked documents. It misses cases where the user phrases the query differently from how the corpus phrases the answer (a vocabulary gap that embeddings sometimes do not bridge). It misses cases where the model needs to verify a claim or fetch supporting evidence after starting its answer.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Perplexity research mode runs agentic RAG: the model decides which sources to search, reads results, decides whether to search again, and iterates until it has enough evidence for the answer.
- ChatGPT browsing and Claude with web search both implement agentic RAG: a web-search tool is exposed to the model, which calls it as many times as needed during the task.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does agentic RAG handle multi-hop questions better than single-shot RAG?
A multi-hop question needs information that links across several documents. Single-shot retrieves top-K chunks for the original query, which usually contains the first hop but rarely the chained ones. Agentic RAG retrieves the first hop, reads it, generates a follow-up query for the second hop, retrieves that, and continues until the chain is closed.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Describing agentic RAG as RAG with reranking or with better embeddings. The defining feature is that retrieval is a runtime tool call the agent decides to make, not a fixed pre-processing step.
60 second bullets to scan on the way to the call.
Define agentic RAG as retrieval inside an agent loop with the model choosing when and what to query.
Define single-shot RAG as one retrieval before generation, with no model decision on retrieval count.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.