Explain the difference between naive single shot RAG and agentic RAG. What capabilities does putting the retrieval step inside the agent loop enable that static retrieval cannot?
Free · 2 AI evals / day
**Single shot RAG** is the simple version. You embed the user's question, find the top matching chunks from a vector store, paste them into the prompt, and ask the model to answer. Retrieval happens exactly once, before generation starts.
That works well when the answer sits in one place. It breaks when the first search misses, or when the question needs two lookups that depend on each other.
**Agentic RAG** fixes this by turning retrieval into a tool the agent can call inside its loop. Instead of one fixed search, the model reasons, decides it needs information, issues a search, reads the result, and then decides whether it needs another search.
This unlocks three things. The agent can do multi-hop lookups, where the first result tells it what to search for next. It can rewrite a vague query into a sharper one after seeing weak results. And it can check whether the retrieved text actually answers the question before committing.
The price is real. Every extra retrieval is another model call, so agentic RAG is slower and more expensive than the single shot version.
**Single shot RAG** retrieves once. You embed the question, pull the top-k chunks, inject them, and generate. The retrieval policy is fixed and blind to what the model later realises it needs.
Agentic RAG instead exposes retrieval as a tool call inside the agent loop. The model decides when to retrieve, what query to issue, and how many times to search, all conditioned on its current reasoning state. Retrieval becomes a decision, not a preprocessing step.
That shift enables three capabilities single shot RAG cannot offer. Multi-hop retrieval means the first result reveals an entity, and the agent issues a follow up query that depends on it. Query rewriting means weak or too broad chunks trigger a sharper second query. Self-checking means the agent inspects whether retrieved evidence supports an answer before generating, and re-retrieves if not.
Agentic RAG wins when a question needs more than one lookup or when the right query is not knowable up front. It loses on simple lookups. Each loop turn is a full model round trip, so latency and token cost grow with the number of hops. Use single shot RAG when one retrieval suffices.
Quick reference
TL;DR
Single shot RAG retrieves once up front; agentic RAG makes retrieval a tool the agent calls repeatedly, deciding when and what to fetch as its reasoning unfolds.
Imagine answering a hard question with a library. Single shot RAG is like grabbing the first three books that match your question, then writing your answer using only those, no second trip allowed. If the books are wrong or too vague, you are stuck. Agentic RAG is like a curious researcher. You read one book, notice it mentions an author you need, walk back to the shelf, and pull that author's work too. If your first search was too broad, you go back and search again with better words. You keep fetching, reading, and deciding what to look up next until you actually have what you need. The trade off is that every extra trip to the shelf costs time, so you only do it when one trip is not enough.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
The answer reliably sits in a single chunk reachable by one embedding of the question. The extra loop turns add latency and token cost while buying no accuracy.
A tight per-request latency budget, for example an interactive chat under one second. Each hop adds a full model round trip, so multi-hop loops blow past tight SLAs.
High query volume where per-query cost must stay minimal. Re-sending a growing transcript every turn multiplies token spend per request.
Strict reproducibility or auditability is required for every retrieval. A dynamic, model-driven retrieval path is harder to certify than a fixed one.
Quick scan · 2–4 min
Everything important, quickly.
Contrast static single shot retrieval with retrieval as tool call, name the three capabilities the loop unlocks (multi-hop, query rewriting, evidence self-checking), then close with the latency and cost price and the router pattern that decides which path a query takes.
Real-world examples
Real products, models, and research that use this idea.
1Perplexity's research mode runs an agentic retrieval loop: it searches, reads, reformulates queries, and searches again before composing a cited answer.
2LangGraph's self-RAG and corrective-RAG templates grade retrieved chunks for relevance and trigger a query rewrite plus re-retrieval when the grade is low.
3LlamaIndex agentic retrievers expose query engines as tools so an agent built on Claude Opus 4.7 or GPT-5.5 can decide which index to hit and how many follow up hops to run.
4Claude Code searches a codebase agentically: it greps, reads a file, then issues new searches based on imports it discovers rather than one fixed embedding lookup.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide at request time whether a query should go down the single shot or the agentic path?
Sign in or go Pro to see the approach for this follow-up.
QWhat stops an agentic RAG loop from re-retrieving forever on a question its corpus cannot answer?
Sign in or go Pro to see the approach for this follow-up.
QHow do you keep context clean when the agent makes many retrieval calls in one run?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Pitching agentic RAG as a strict upgrade over single shot RAG. It is not. For one hop questions the loop just adds latency and cost for no accuracy gain.