Explain the difference between naive single shot RAG and agentic RAG. What capabilities does putting the retrieval step inside the agent loop enable that static retrieval cannot?
Single shot RAG retrieves once up front; agentic RAG makes retrieval a tool the agent calls repeatedly, deciding when and what to fetch as its reasoning unfolds.
Imagine answering a hard question with a library. Single shot RAG is like grabbing the first three books that match your question, then writing your answer using only those, no second trip allowed. If the books are wrong or too vague, you are stuck. Agentic RAG is like a curious researcher. You read one book, notice it mentions an author you need, walk back to the shelf, and pull that author's work too. If your first search was too broad, you go back and search again with better words. You keep fetching, reading, and deciding what to look up next until you actually have what you need. The trade off is that every extra trip to the shelf costs time, so you only do it when one trip is not enough.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Agentic RAG is what you get when you stop treating retrieval as a fixed preprocessing step and start treating it as an action the model can take whenever it decides it needs information. The contrast with naive single shot RAG is the cleanest way to see why the agent loop matters for retrieval at all.
In single shot RAG the pipeline is linear: embed the question, fetch the top-k chunks, inject them into the prompt, and generate once. The retrieval policy is fixed before the model ever reasons, and the model has no way to ask for more. In agentic RAG, retrieval is exposed as a tool inside the loop, so the model chooses when to call it, what query to send, and when it has gathered enough to answer.
The structural difference: where the retrieval policy lives
In single shot RAG the retrieval policy is static and lives outside the model. The system embeds the question, runs a nearest-neighbour search, takes the top-k results, and pastes them in. The retriever never sees the model's reasoning, and the model never gets a second chance to influence what it receives. The entire retrieval decision is made before a single token of reasoning has happened, on the strength of one embedding of the raw question.
The canonical single shot scoring step is just a similarity lookup against the indexed chunks:
That single vector has to encode the full intent of the question, and the top-k cut has to be wide enough to catch the answer yet narrow enough to avoid drowning the prompt in noise. When the question is simple, this works beautifully and is hard to beat on cost. When the question is compound or vaguely worded, the averaged embedding lands between the relevant regions of the index and the retrieved chunks miss.
In agentic RAG that same lookup becomes a tool the model can invoke many times with different queries. The policy is now dynamic: the model conditions each retrieval on everything it has read so far. Retrieval stops being a preprocessing step and becomes a decision the agent makes mid-reasoning, exactly like any other tool call in an agent loop. The same observe, reason, act, observe cycle that defines an agent now wraps the retriever, and the model's intermediate conclusions steer where it looks next.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Perplexity's research mode runs an agentic retrieval loop: it searches, reads, reformulates queries, and searches again before composing a cited answer.
- LangGraph's self-RAG and corrective-RAG templates grade retrieved chunks for relevance and trigger a query rewrite plus re-retrieval when the grade is low.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide at request time whether a query should go down the single shot or the agentic path?
Add a lightweight classifier or router that scores query complexity, such as whether it is multi-hop or comparative. Route simple factoids to single shot RAG and reserve the loop for queries that score as multi-hop or research-style.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Pitching agentic RAG as a strict upgrade over single shot RAG. It is not. For one hop questions the loop just adds latency and cost for no accuracy gain.
60 second bullets to scan on the way to the call.
Define single shot RAG as a fixed retrieve once then generate pipeline.
Define agentic RAG as retrieval exposed as a tool call inside the loop.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.