What problem does LlamaIndex's `RouterQueryEngine` solve that a single `QueryEngine` cannot?

MCQ·Medium·4.0 · 0·~1 min·Asked atMicrosoft Databricks Perplexity

Attempt it

A `QueryEngine` in LlamaIndex retrieves and synthesizes over one Index. Typically a vector index, sometimes a summary or SQL index. It is great when every query needs the same kind of retrieval. Real workloads do not always look like that. Some questions need a summary of a whole document. Some need a lookup over structured data. Some need semantic search over chunks. One Index cannot serve all three well. A vector index is bad at summarisation, a summary index is bad at lookup, a SQL index does not handle free-form semantic queries. `RouterQueryEngine` solves this by wrapping multiple QueryEngines (one per shape of question) plus a Selector. For each incoming query, the Selector picks the sub-engine best suited to answer it and forwards the query. The user sees one entrypoint; behind it, three completely different retrieval strategies run depending on what the question needs. The other three options describe optimisations to a single retrieval path, batching, caching, re-ranking, that are handled by different LlamaIndex primitives (ingestion config, retriever wrappers, NodePostprocessors), not by RouterQueryEngine.

**RouterQueryEngine is a multi-index dispatch primitive.** It wraps a list of QueryEngines, each backed by an index suited to a different question shape (summary, vector, SQL, keyword), plus a Selector that picks one of them per query. **Why it exists.** A single Index has a sweet spot. A vector index over chunks is great for 'what does the author say about X' semantic search but bad for 'summarise this document' (chunks lose document structure) and bad for 'how many rows match X' (no structured query surface). A summary index is great for whole-document questions but loses fine-grained retrieval. A SQL index handles structured queries but cannot do free-form semantic search. RouterQueryEngine lets one application surface three or four Indexes behind one entrypoint and dispatch correctly. **How the Selector works.** Two common implementations: LLM-based (the classifier reads the query and picks an engine by name, often with a chain-of-thought rationale) and embedding-based (the query is embedded and compared to descriptions of each sub-engine). LLM-based is more flexible and more expensive; embedding-based is faster and cheaper but less nuanced. **Why the distractors are wrong.** Batching identical queries is an ingestion or caching concern; LlamaIndex handles it via the underlying vector client. Caching retrieved Nodes is a NodePostprocessor or wrapper concern. Cross-encoder re-ranking is a NodePostprocessor (`SentenceTransformerRerank`, `CohereRerank`). All three are optimisations on a single retrieval path; only the correct option describes choosing among paths.

Quick reference

TL;DR

RouterQueryEngine picks among several sub-engines (vector, SQL, summary) per query so one front door can serve structurally different question types.

Memory aid

Easy to grasp

Picture the front desk at a hospital. A single QueryEngine is one specialist with one toolkit. Great for the questions they know, useless for the rest. RouterQueryEngine is a triage nurse. You arrive with any question; she listens, decides whether you need radiology, the pharmacy, or general practice, and sends you to the right room. The departments have completely different tools, X-ray machines, pill counters, stethoscopes, but the patient only sees one front door. The router does not retrieve anything itself. It picks the engine that should retrieve, and that engine handles the rest.

Key concepts

RAG LLM Frameworks

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 min: why one Index cannot serve all shapes, how the Selector picks, what differentiates LLM-based from embedding-based, and how to calibrate sub-engine descriptions.

Real-world examples

Real products, models, and research that use this idea.

LlamaIndex's official tutorials use RouterQueryEngine to combine a summary index and a vector index over the same documents for 'summarise the report' vs 'what does it say about X' questions.
Notion AI's document Q&A combines whole-document summarisation, page-level retrieval, and structured property queries. Exactly the heterogeneous workload RouterQueryEngine targets.
Replit's repo assistant routes 'what does this function do' (semantic) vs 'find all callers of X' (graph / structured). Different retrieval shapes behind one front door.
LangChain's MultiRetrievalQAChain and RouterRunnable solve similar dispatch patterns; the underlying problem (one Index per shape) is framework-agnostic.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you compare RouterQueryEngine to a SubQuestionQueryEngine?

QWhat does the Selector's prompt look like in practice?

QHow would you A/B test a new sub-engine without breaking production?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Building one fat vector index for every kind of question, summarisation, lookup, structured query, and getting bad answers because the index shape only fits one question type.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.