Match LlamaIndex Response Synthesizers to use cases

Match pairs·Medium·4.0 · 0·~2 min·Asked atMicrosoft Databricks Perplexity

Attempt it

Iterates through retrieved nodes, refining a running answer one node at a time. Best when you need every node's contribution but accept N sequential LLM calls

Recursively summarizes node groups into intermediate summaries, then summarizes those. Best for long-document summarization that does not fit any single prompt

Calls the LLM on each node independently and returns the concatenated answers. Best when each node's answer is independent and you want a per-source response

Stuffs as many retrieved nodes as fit into one prompt and answers in a single call. Best when retrieval is small enough to fit the context window

Quick reference

TL;DR

Four synthesizers, four shapes: compact (one call), refine (sequential improve), tree_summarize (recursive merge), accumulate (one answer per node).

Memory aid

Easy to grasp

Think of writing a report from a stack of notes. Compact: skim the stack at once and write the report in one sitting. Refine: read one note, write a draft, read the next note, improve the draft, repeat. Tree-summarize: split the stack into piles, write a mini-summary per pile, then summarize the summaries. Accumulate: write a separate paragraph about each note and staple them together. Same stack, four very different writing styles. And the right pick depends on how big the stack is and what shape of report you need.

Key concepts

LLM Frameworks Retrieval-Augmented Generation

Detailed answer & concept explanation

~4 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

6 min: four synthesizers, call shapes, when each wins, production guard against silent truncation, when to write a custom synthesiser.

Code example

python

from llama_index.core import VectorStoreIndex, get_response_synthesizer
from llama_index.core.response_synthesizers import ResponseMode

index = VectorStoreIndex.from_documents(docs)

# Compact: cheap default, single LLM call
compact_qe = index.as_query_engine(response_mode=ResponseMode.COMPACT)

# Tree summarize: hierarchical merge for long-context summarisation
tree_qe = index.as_query_engine(response_mode=ResponseMode.TREE_SUMMARIZE)

# Accumulate: per-node answers concatenated
acc_qe = index.as_query_engine(response_mode=ResponseMode.ACCUMULATE)

# Refine: sequential improvement across nodes
refine_qe = index.as_query_engine(response_mode=ResponseMode.REFINE)

Side-by-side

Synthesizer	LLM calls	Aggregation shape	Best fit
compact	1	Stuff and answer	Short-context QA, cheap default
refine	N (sequential)	Incremental improvement	Ordered nodes, each adds a refinement
tree_summarize	O(log N) depth	Hierarchical merge	Long-document summarisation that overflows window
accumulate	N (parallelisable)	Independent per-node answers	Per-source citation, comparative review

Real-world examples

Real products, models, and research that use this idea.

Notion AI's enterprise document search uses compact plus rerank for most QA and tree_summarize for executive summaries across multi-document corpora
Legal-tech tools like Harvey use accumulate when each cited authority must produce its own grounded paragraph in the response
Long-PDF summarisation features (e.g. Mendeley, Scite) lean on tree_summarize because individual papers overflow any single prompt window

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhen would you write a custom ResponseSynthesizer rather than pick one of the four?

QHow does tree_summarize parallelise across the tree?

QWhy does refine sometimes produce worse answers than compact even on the same retrieval?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Defaulting to compact for every query. The cheap default fails the moment retrieval exceeds the context window, and silently truncates context for the harder questions where context matters most.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.