Trace an adaptive RAG gate's retrieve or skip decision across four incoming queries
An adaptive RAG router retrieves only when a query needs external, up to date, or corpus specific knowledge; otherwise it answers directly from the model's parameters. The knowledge base is a company's internal docs. Decide RETRIEVE or SKIP for each query, in order: (1) "Hi, can you help me?" (2) "What is 12 multiplied by 9?" (3) "What is our current PTO carryover policy?" (4) "Summarize the latest changes in our deployment runbook."
An adaptive RAG gate retrieves only for corpus-specific or time-sensitive queries: greeting and arithmetic SKIP, internal-policy and recent-changes RETRIEVE.
Imagine a helpful coworker who keeps a binder of company rules on their desk. If you say "hi" or ask "what's 12 times 9," they answer straight away — no need to flip through the binder. But if you ask "what's our PTO carryover policy," they reach for the binder, because that fact only lives there. Adaptive RAG, short for Retrieval-Augmented Generation, gives the model that same judgment: open the binder only when the answer actually lives in it, and answer from memory when it doesn't.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Naive RAG retrieves on every single query. That sounds safe until you notice it runs a vector search before answering "hello," stuffs irrelevant chunks into a math problem, and pays latency and token cost on questions the model could have answered instantly from its own parameters. Adaptive RAG fixes this by adding a gate: decide, per query, whether retrieval is actually needed.
This question is a clean trace through that gate's logic, with four queries chosen to span the space. The skill being tested isn't memorizing an answer key — it's the underlying classification: which questions does a frozen model already know or compute, and which require facts that live only in a private, current corpus? We'll work through each query, then lift to the real engineering problem — the router is itself fallible, and its two error types cost very differently.
The test that decides every query
An adaptive router answers one question before it does anything else: does answering this well require a fact the model doesn't already have? That splits into two checks.
First, is the answer in the model's parametric memory? General world knowledge and anything computable (arithmetic, simple logic) is — the model can produce it without help. Private, organization-specific facts are not; they were never in the training data.
Second, is the answer static or fresh? Even if a topic is in the corpus, "the latest" or "current" version is time-sensitive, and a frozen model's knowledge has a cutoff. Fresh facts demand retrieval regardless of whether the model has seen something about the topic.
Run both checks and you get a clean rule: retrieve when the answer is non-parametric (private) or fresh; skip when it's parametric and static. The four queries in this question are deliberately one per quadrant — a greeting (social, no facts), arithmetic (parametric, static), a PTO policy (private, static), and the latest runbook changes (private and fresh).
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Self-RAG trains the model to emit a retrieve on demand reflection token, deciding per query whether to fetch documents
- CRAG (Corrective RAG) retrieves, grades the result as correct/ambiguous/incorrect, and falls back to web search when retrieval is wrong
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you actually implement the retrieve or skip decision in production?
Options range from a cheap learned classifier or few-shot LLM router over the query text, to Self-RAG's trained reflection token, to CRAG's retrieve then grade and correct loop. Pick based on latency budget and how often false skips are catastrophic; the router itself should be evaluated on a labeled query set.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Retrieving on every query 'to be safe' — which injects irrelevant chunks into greetings and arithmetic, wasting a vector search and adding distractor noise to questions the model already handles.
60 second bullets to scan on the way to the call.
State the four-query output and the one test you apply to each query
Explain why a greeting and pure arithmetic skip retrieval
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.