Match LlamaIndex Response Synthesizers to use cases
Drag each answer to line up with its matching prompt
Iterates through retrieved nodes, refining a running answer one node at a time. Best when you need every node's contribution but accept N sequential LLM calls
tree_summarize
Recursively summarizes node groups into intermediate summaries, then summarizes those. Best for long document summarization that does not fit any single prompt
accumulate
Calls the LLM on each node independently and returns the concatenated answers. Best when each node's answer is independent and you want a per source response
refine
Stuffs as many retrieved nodes as fit into one prompt and answers in a single call. Best when retrieval is small enough to fit the context window
compact
Four synthesizers, four shapes: compact (one call), refine (sequential improve), tree_summarize (recursive merge), accumulate (one answer per node).
Think of writing a report from a stack of notes. Compact: skim the stack at once and write the report in one sitting. Refine: read one note, write a draft, read the next note, improve the draft, repeat. Tree-summarize: split the stack into piles, write a mini-summary per pile, then summarize the summaries. Accumulate: write a separate paragraph about each note and staple them together. Same stack, four very different writing styles. And the right pick depends on how big the stack is and what shape of report you need.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
RAG is rarely lost on retrieval. By the time top-k similarity returns the right chunks, the question is what to do with them. Stuff them all into one prompt? Walk them sequentially and refine? Summarise them hierarchically? Answer each independently?
LlamaIndex bundles those four answers as ResponseSynthesizers. Compact, refine, tree_summarize, accumulate. Each encodes a legitimate aggregation pattern with different cost, latency, and quality tradeoffs. Picking the wrong one is the difference between a 1-call $0.002 query and a 12-call $0.024 query that produces a worse answer.
The matching question separates candidates who have read the LlamaIndex tutorial from candidates who have shipped RAG.
Compact. Stuff and answer
Compact concatenates retrieved nodes (delimited, with citation markers) and stuffs them into a single prompt that asks the model to answer the query. One LLM call.
It is the cheap default. For top-k = 5 with 300-token chunks, total context is around 1500 tokens. Easily fits any modern model. The answer arrives in one round-trip.
The failure mode is silent truncation. When retrieval overflows the context window, compact drops nodes to fit. The user sees a confident answer that may have lost the most relevant citation. Production posture: log the token count, set a guard that switches to refine or tree_summarize when context would overflow.
Compact is the right default for short-context QA. It is the wrong default for workloads where retrieval depth varies wildly per query.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
from llama_index.core import VectorStoreIndex, get_response_synthesizer
from llama_index.core.response_synthesizers import ResponseMode
index = VectorStoreIndex.from_documents(docs)
# Compact: cheap default, single LLM call
compact_qe = index.as_query_engine(response_mode=ResponseMode.COMPACT)
# Tree summarize: hierarchical merge for long-context summarisation
tree_qe = index.as_query_engine(response_mode=ResponseMode.TREE_SUMMARIZE)
# Accumulate: per-node answers concatenated
acc_qe = index.as_query_engine(response_mode=ResponseMode.ACCUMULATE)
# Refine: sequential improvement across nodes
refine_qe = index.as_query_engine(response_mode=ResponseMode.REFINE)
| Synthesizer | LLM calls | Aggregation shape | Best fit |
|---|---|---|---|
| compact | 1 | Stuff and answer | Short-context QA, cheap default |
| refine | N (sequential) | Incremental improvement | Ordered nodes, each adds a refinement |
| tree_summarize | O(log N) depth | Hierarchical merge | Long-document summarisation that overflows window |
| accumulate | N (parallelisable) | Independent per-node answers | Per-source citation, comparative review |
Real products, models, and research that use this idea.
- Notion AI's enterprise document search uses compact plus rerank for most QA and tree_summarize for executive summaries across multi-document corpora
- Legal-tech tools like Harvey use accumulate when each cited authority must produce its own grounded paragraph in the response
What an interviewer would ask next. Try answering before peeking at the approach.
QWhen would you write a custom ResponseSynthesizer rather than pick one of the four?
Cover cases like structured-output synthesis, citation-bound formatting, or multi-step refinement with intermediate validation that none of the four off the shelf modes cover.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Defaulting to compact for every query. The cheap default fails the moment retrieval exceeds the context window, and silently truncates context for the harder questions where context matters most.
60 second bullets to scan on the way to the call.
Call shape of each synthesiser
When compact silently truncates and how to guard against it
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.