How does LlamaIndex ChatEngine differ from QueryEngine?

MCQ·Medium·4.0 · 0·~1 min·Asked atMicrosoft Databricks Perplexity

Attempt it

LlamaIndex has two related abstractions on top of an Index: QueryEngine and ChatEngine. QueryEngine is the stateless RAG primitive. You hand it a query, it retrieves the most relevant Nodes from the Index, a response synthesizer composes them into an answer, and the call returns. No memory of previous queries, no notion of a conversation. Each call is independent. ChatEngine wraps a QueryEngine (or directly wraps an Index) and adds two things: conversation memory and a strategy for handling follow-up turns. The strategy depends on the mode you pick: - `condense_question`. Rewrites a follow-up into a standalone question using the chat history, then sends that standalone question to the underlying retrieve and synthesize step. - `context`. Keeps the conversation history in the prompt and lets the model reason over both the history and the retrieved context. - `react`. Runs an agent loop that may or may not retrieve, depending on the model's tool-calling decision. The key difference is multi-turn awareness, not the index type, not streaming, and not the provider.

**QueryEngine is stateless RAG; ChatEngine adds conversation handling.** Both sit on top of an Index, both can use the same retriever and response synthesizer, but they answer different questions. `QueryEngine.query(q)` is one-shot: retrieve relevant Nodes for `q`, synthesize an answer, return. No memory of prior calls. This is the right primitive for a search box or a single-turn Q&A endpoint. `ChatEngine.chat(message)` is multi-turn: maintain the conversation history, decide how to handle a follow-up, and only then drop into the retrieve and synthesize layer. The 'how to handle the follow-up' part is what the modes encode: - **condense_question**. Uses the chat history to rewrite an underspecified follow-up ('and what about its side effects?') into a standalone query ('what are the side effects of ibuprofen?') before retrieval. Solves the pronoun with no antecedent retrieval bug. - **context**. Injects the chat history into the synthesis prompt alongside the retrieved context, letting the model reason over both. Cheaper than condense_question (no extra rewrite call) but worse retrieval when the follow-up is underspecified. - **react**. Runs a tool-calling loop where retrieval is one tool the model can choose to call. Right when some turns need retrieval and others (small talk, clarifications) do not. The other three options are wrong on the facts. ChatEngine uses whatever Index you give it; both stream; both work with any provider LlamaIndex supports.

Quick reference

TL;DR

QueryEngine is stateless retrieve and synthesize; ChatEngine adds conversation memory and can rewrite underspecified follow-ups into standalone queries before retrieval.

Memory aid

Easy to grasp

Picture a research assistant at a library. The QueryEngine version is the desk attendant. You walk up, ask one question, they pull a book, read you the relevant page, you leave. Every visit starts fresh; they do not remember you. The ChatEngine version is a personal researcher you book by the hour. They remember what you asked last time. When you say 'and what about its side effects?', they know 'its' means the drug you discussed three minutes ago, and they rewrite the question into something they can actually search before pulling the next book. The difference is memory and follow-up rewriting, not the books or the search.

Key concepts

RAG LLM Frameworks

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

4 min: QueryEngine vs ChatEngine, the three ChatEngine modes, the antecedent-resolution failure mode, and how to pick a mode.

Side-by-side

Axis	QueryEngine	ChatEngine
State	Stateless per call	Conversation memory across turns
Turn handling	Literal query → retrieve → synthesize	Mode-dependent (condense / context / react)
Follow-up rewriting	No	Yes in condense_question mode
Tool-calling option	No (just retrieve)	Yes in react mode
Cost per turn	1 retrieval + 1 synthesis	Up to 1 rewrite + 1 retrieval + 1 synthesis
Best fit	Search box, single-turn Q&A	Multi-turn chatbot with referential follow-ups

Real-world examples

Real products, models, and research that use this idea.

LlamaIndex's documentation chatbot uses ChatEngine in `condense_question` mode because doc questions chain into 'how does this interact with X' follow-ups.
Notion AI and Linear's assistant patterns rely on the agentic mode equivalent. Retrieval as a tool, not as the default action per turn.
QueryEngine alone backs many 'search-over-PDFs' single-turn demos where each query is independent.
LangChain's ConversationalRetrievalChain solves the same problem with a similar two-step rewrite then retrieve flow under different naming.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you decide between condense_question and react?

QWhat happens to the condensed query in condense_question mode if the history is misleading?

QCould you implement condense_question yourself on top of QueryEngine?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Using QueryEngine for a chatbot and discovering that follow-up questions like 'and what about its side effects?' retrieve garbage because the pronoun gets embedded with no antecedent.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.