A `QueryEngine` in LlamaIndex retrieves and synthesizes over one Index. Typically a vector index, sometimes a summary or SQL index. It is great when every query needs the same kind of retrieval.
Real workloads do not always look like that. Some questions need a summary of a whole document. Some need a lookup over structured data. Some need semantic search over chunks. One Index cannot serve all three well. A vector index is bad at summarisation, a summary index is bad at lookup, a SQL index does not handle free-form semantic queries.
`RouterQueryEngine` solves this by wrapping multiple QueryEngines (one per shape of question) plus a Selector. For each incoming query, the Selector picks the sub-engine best suited to answer it and forwards the query. The user sees one entrypoint; behind it, three completely different retrieval strategies run depending on what the question needs.
The other three options describe optimisations to a single retrieval path, batching, caching, re-ranking, that are handled by different LlamaIndex primitives (ingestion config, retriever wrappers, NodePostprocessors), not by RouterQueryEngine.
**RouterQueryEngine is a multi-index dispatch primitive.** It wraps a list of QueryEngines, each backed by an index suited to a different question shape (summary, vector, SQL, keyword), plus a Selector that picks one of them per query.
**Why it exists.** A single Index has a sweet spot. A vector index over chunks is great for 'what does the author say about X' semantic search but bad for 'summarise this document' (chunks lose document structure) and bad for 'how many rows match X' (no structured query surface). A summary index is great for whole-document questions but loses fine-grained retrieval. A SQL index handles structured queries but cannot do free-form semantic search. RouterQueryEngine lets one application surface three or four Indexes behind one entrypoint and dispatch correctly.
**How the Selector works.** Two common implementations: LLM-based (the classifier reads the query and picks an engine by name, often with a chain-of-thought rationale) and embedding-based (the query is embedded and compared to descriptions of each sub-engine). LLM-based is more flexible and more expensive; embedding-based is faster and cheaper but less nuanced.
**Why the distractors are wrong.** Batching identical queries is an ingestion or caching concern; LlamaIndex handles it via the underlying vector client. Caching retrieved Nodes is a NodePostprocessor or wrapper concern. Cross-encoder re-ranking is a NodePostprocessor (`SentenceTransformerRerank`, `CohereRerank`). All three are optimisations on a single retrieval path; only the correct option describes choosing among paths.
Quick reference
TL;DR
RouterQueryEngine picks among several sub-engines (vector, SQL, summary) per query so one front door can serve structurally different question types.
Picture the front desk at a hospital. A single QueryEngine is one specialist with one toolkit. Great for the questions they know, useless for the rest.
RouterQueryEngine is a triage nurse. You arrive with any question; she listens, decides whether you need radiology, the pharmacy, or general practice, and sends you to the right room. The departments have completely different tools, X-ray machines, pill counters, stethoscopes, but the patient only sees one front door.
The router does not retrieve anything itself. It picks the engine that should retrieve, and that engine handles the rest.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Workload where every query has the same shape Single QueryEngine is simpler and cheaper; RouterQueryEngine adds a Selector call for routing that always resolves the same way.
Latency-critical endpoint with no slack for a classifier hop Even embedding-based Selectors add a vector-similarity call; if your p99 is already tight, route at the application layer with explicit endpoints.
Two indexes that retrieve from the same underlying corpus with nearly identical descriptions The Selector cannot distinguish them reliably; collapse to one engine or differentiate the descriptions sharply.
Teams that prefer explicit per-intent endpoints over a black-box classifier Routing as API (one endpoint per question type) is more debuggable and gives the front end direct control over which backend runs.
Quick scan · 2–4 min
Everything important, quickly.
4 min: why one Index cannot serve all shapes, how the Selector picks, what differentiates LLM-based from embedding-based, and how to calibrate sub-engine descriptions.
Real-world examples
Real products, models, and research that use this idea.
1LlamaIndex's official tutorials use RouterQueryEngine to combine a summary index and a vector index over the same documents for 'summarise the report' vs 'what does it say about X' questions.
2Notion AI's document Q&A combines whole-document summarisation, page-level retrieval, and structured property queries. Exactly the heterogeneous workload RouterQueryEngine targets.
3Replit's repo assistant routes 'what does this function do' (semantic) vs 'find all callers of X' (graph / structured). Different retrieval shapes behind one front door.
4LangChain's MultiRetrievalQAChain and RouterRunnable solve similar dispatch patterns; the underlying problem (one Index per shape) is framework-agnostic.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you compare RouterQueryEngine to a SubQuestionQueryEngine?
Sign in or go Pro to see the approach for this follow-up.
QWhat does the Selector's prompt look like in practice?
Sign in or go Pro to see the approach for this follow-up.
QHow would you A/B test a new sub-engine without breaking production?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Building one fat vector index for every kind of question, summarisation, lookup, structured query, and getting bad answers because the index shape only fits one question type.