What problem does LlamaIndex's RouterQueryEngine solve that a single QueryEngine cannot?

MCQ·Medium·4.0 · 0·~1 min·Asked atHaptik Together Ai

Attempt it

A `QueryEngine` in LlamaIndex retrieves and synthesizes over one Index, typically a vector index and sometimes a summary or SQL index. It is great when every query needs the same kind of retrieval. Real workloads do not always look like that. Some questions need a summary of a whole document. Some need a lookup over structured data. Some need semantic search over chunks. One Index cannot serve all three well. A vector index is bad at summarisation, and a summary index is bad at lookup. A SQL index does not handle free-form semantic queries. `RouterQueryEngine` solves this by wrapping multiple QueryEngines (one per shape of question) plus a Selector. For each incoming query, the Selector picks the sub-engine best suited to answer it and forwards the query. The user sees one entrypoint. Behind it, three completely different retrieval strategies run depending on what the question needs. The other three options describe optimisations to a single retrieval path (batching, caching, re-ranking). Different LlamaIndex primitives handle those (ingestion config, retriever wrappers, NodePostprocessors), not RouterQueryEngine.

**RouterQueryEngine is a multi-index dispatch primitive.** It wraps a list of QueryEngines, each backed by an index suited to a different question shape (summary, vector, SQL, keyword), plus a Selector that picks one of them per query. **Why it exists.** Each Index suits one kind of question. A vector index over chunks is great for 'what does the author say about X' semantic search. It is bad for 'summarise this document' (chunks lose document structure) and for 'how many rows match X' (no structured query surface). A summary index is great for whole-document questions but loses fine-grained retrieval. A SQL index handles structured queries but cannot do free-form semantic search. RouterQueryEngine lets one application surface three or four Indexes behind one entrypoint and dispatch correctly. **How the Selector works.** There are two common implementations: LLM-based (the classifier reads the query and picks an engine by name, often with a chain-of-thought rationale) and embedding-based (the query is embedded and compared to descriptions of each sub-engine). LLM-based is more flexible and more expensive. Embedding-based is faster and cheaper but less nuanced. **Why the other options are wrong.** Batching identical queries is a caching concern outside the router. Caching retrieved Nodes is a NodePostprocessor or wrapper concern. Cross-encoder re-ranking is a NodePostprocessor (`SentenceTransformerRerank`, `CohereRerank`). All three are optimisations on a single retrieval path. Only the correct option describes choosing among paths.

TL;DR

RouterQueryEngine picks among several sub-engines (vector, SQL, summary) per query so one entry point can serve structurally different question types.

Memory aid

Easy to grasp

Picture the front desk at a hospital. A single QueryEngine is one specialist with one toolkit. They are great for the questions they know and useless for the rest. RouterQueryEngine is a triage nurse. You arrive with any question. She listens, decides whether you need radiology, the pharmacy, or general practice, and sends you to the right room. The departments have completely different tools (X-ray machines, pill counters, stethoscopes), but the patient only sees one front door. The router does not retrieve anything itself. It picks the engine that should retrieve, and that engine handles the rest.

Key concepts

RAG LLM Frameworks

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

RAG starts simple: one corpus, one vector index, one retriever. It stays simple until the questions stop looking alike. As soon as the workload mixes 'summarise this report', 'what does the policy say about overtime', and 'how many incidents matched severity 3', a single Index reaches its structural limit. Each question shape needs a different Index shape, and no single Index wins all three.

RouterQueryEngine is LlamaIndex's primitive for this: it composes several engines instead of forcing one engine to handle every shape.

Why one Index shape cannot serve all question shapes

Each Index type in LlamaIndex suits a different question shape.

Vector indexes (Pinecone, Weaviate, FAISS, Chroma) embed document chunks and answer 'what does the corpus say about X' via similarity search. Strong on semantic recall over passages. Weak on whole-document questions because chunks lose document-level signal, and weak on structured filter queries.
Summary indexes (SummaryIndex) store a document's nodes as a list and pass all of them to the synthesizer at query time. Strong on 'summarise the report' or 'what is the gist of this section'. Weak on fine-grained passage retrieval because every query reads the whole document instead of the best-matching passages.
SQL / structured indexes sit on top of a database or DataFrame. Strong on 'how many rows where status = X'. Useless for free-form semantic search because rows are not embeddings.
Keyword / BM25 indexes answer exact-token questions. Strong when the user knows the term; weak when they paraphrase.

A mixed workload forces a choice: build one Index that does everything badly, or build several Indexes that each do their shape well. RouterQueryEngine is the way to do the second without exposing the complexity to the application's caller.

The Selector: the component that decides

Descriptions as the tuning surface

How RouterQueryEngine fits the rest of the stack

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

LlamaIndex's official tutorials use RouterQueryEngine to combine a summary index and a vector index over the same documents for 'summarise the report' vs 'what does it say about X' questions.
An internal help-desk assistant that sends 'summarise this policy' to a summary index and 'how many tickets were filed last week' to a SQL engine is the mixed workload RouterQueryEngine targets.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you compare RouterQueryEngine to a SubQuestionQueryEngine?

SubQuestionQueryEngine decomposes a complex query into sub-questions and answers each against potentially different engines, then synthesizes. RouterQueryEngine picks one engine for the whole query. The first composes, the second selects.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Building one fat vector index for every kind of question (summarisation, lookup, structured query) and getting bad answers because the index shape only fits one question type.

60 second bullets to scan on the way to the call.

One Index shape per question shape: the structural limit of a single QueryEngine
RouterQueryEngine as dispatch across several sub-engines

Primary sources. Browse if you want the original framing.