Iterates through retrieved nodes, refining a running answer one node at a time. Best when you need every node's contribution but accept N sequential LLM calls
Recursively summarizes node groups into intermediate summaries, then summarizes those. Best for long-document summarization that does not fit any single prompt
Calls the LLM on each node independently and returns the concatenated answers. Best when each node's answer is independent and you want a per-source response
Stuffs as many retrieved nodes as fit into one prompt and answers in a single call. Best when retrieval is small enough to fit the context window
**LlamaIndex** ships four response-synthesis modes that decide how retrieved nodes turn into a final answer.
**compact** stuffs as many nodes as fit into one prompt and answers in a single LLM call. It is the cheap default. One call, low cost, fast. It works when retrieval is small enough to fit the context window.
**refine** processes nodes sequentially: the first node produces an initial answer, each subsequent node refines that answer. The cost is N sequential LLM calls. It works when every node's contribution matters and the running answer needs to improve with each one.
**tree_summarize** recursively summarises node groups into intermediate summaries, then summarises those. It is the right pick for long-document summarisation that does not fit any single prompt. Log(N) depth instead of linear refine.
**accumulate** calls the LLM on each node independently and returns the concatenated answers. It is the right pick when you want a per-source response. Citation-heavy QA, comparative reading, multi-document review where each answer should stand alone.
The matching follows the shape: linear improvement → refine; hierarchical merge → tree_summarize; per-node answer → accumulate; single-shot → compact.
**LlamaIndex** ships four ResponseSynthesizers, each with a distinct call shape.
- **compact**. Stuff as many retrieved nodes as fit into one prompt, answer in a single LLM call. Cheap default. Fails silently when retrieval overflows the window unless you've set a token-limit guard.
- **refine**. Sequential: answer over node 1, then refine that answer with node 2, then with node 3, and so on. N sequential LLM calls. Right when every node matters and the running answer should improve incrementally.
- **tree_summarize**. Hierarchical: group nodes, summarise each group, then summarise the summaries until one answer remains. log(N)-depth call tree. Right for long-document summarisation.
- **accumulate**. One LLM call per node, returns the concatenation. Right for per-source answers. Comparative reading, citation-heavy QA, multi-tenant review.
The matching follows two axes. Call count (compact = 1, refine = N sequential, tree = log N parallel-ish, accumulate = N parallel) and aggregation shape (compact = stuff, refine = improve, tree = merge, accumulate = independent). Pick by which axis matters: cost (compact), incremental refinement (refine), context overflow (tree_summarize), or per-source independence (accumulate).
Quick reference
TL;DR
Four synthesizers, four shapes: compact (one call), refine (sequential improve), tree_summarize (recursive merge), accumulate (one answer per node).
Think of writing a report from a stack of notes. Compact: skim the stack at once and write the report in one sitting. Refine: read one note, write a draft, read the next note, improve the draft, repeat. Tree-summarize: split the stack into piles, write a mini-summary per pile, then summarize the summaries. Accumulate: write a separate paragraph about each note and staple them together. Same stack, four very different writing styles. And the right pick depends on how big the stack is and what shape of report you need.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Single retrieved chunk per query (top-k = 1) Synthesiser choice does not matter at one node; compact is the only call shape that makes sense.
Workloads where structured output (JSON, Pydantic) is required ResponseSynthesizers are free-form text; use a separate output parser or structured-output synthesiser instead.
Latency-critical single-turn QA with strict P95 budgets Refine and accumulate burn N LLM calls; only compact meets a one-call latency budget.
Workload better expressed as the new LlamaIndex Workflow over Steps Workflows generalise synthesiser logic with typed events; complex synthesis pipelines belong there.
Quick scan · 2–4 min
Everything important, quickly.
6 min: four synthesizers, call shapes, when each wins, production guard against silent truncation, when to write a custom synthesiser.
Code example
python
from llama_index.core import VectorStoreIndex, get_response_synthesizer
from llama_index.core.response_synthesizers import ResponseMode
index = VectorStoreIndex.from_documents(docs)
# Compact: cheap default, single LLM call
compact_qe = index.as_query_engine(response_mode=ResponseMode.COMPACT)
# Tree summarize: hierarchical merge for long-context summarisation
tree_qe = index.as_query_engine(response_mode=ResponseMode.TREE_SUMMARIZE)
# Accumulate: per-node answers concatenated
acc_qe = index.as_query_engine(response_mode=ResponseMode.ACCUMULATE)
# Refine: sequential improvement across nodes
refine_qe = index.as_query_engine(response_mode=ResponseMode.REFINE)
Side-by-side
Synthesizer
LLM calls
Aggregation shape
Best fit
compact
1
Stuff and answer
Short-context QA, cheap default
refine
N (sequential)
Incremental improvement
Ordered nodes, each adds a refinement
tree_summarize
O(log N) depth
Hierarchical merge
Long-document summarisation that overflows window
accumulate
N (parallelisable)
Independent per-node answers
Per-source citation, comparative review
Real-world examples
Real products, models, and research that use this idea.
1Notion AI's enterprise document search uses compact plus rerank for most QA and tree_summarize for executive summaries across multi-document corpora
2Legal-tech tools like Harvey use accumulate when each cited authority must produce its own grounded paragraph in the response
3Long-PDF summarisation features (e.g. Mendeley, Scite) lean on tree_summarize because individual papers overflow any single prompt window
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhen would you write a custom ResponseSynthesizer rather than pick one of the four?
Sign in or go Pro to see the approach for this follow-up.
QHow does tree_summarize parallelise across the tree?
Sign in or go Pro to see the approach for this follow-up.
QWhy does refine sometimes produce worse answers than compact even on the same retrieval?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Defaulting to compact for every query. The cheap default fails the moment retrieval exceeds the context window, and silently truncates context for the harder questions where context matters most.