1QueryEngine. The assembled retrieve then synthesize entrypoint
2Index. Nodes organized into a queryable structure (vector / keyword / tree)
3ResponseSynthesizer. Composes the final answer from retrieved Nodes
4Retriever. Pulls candidate Nodes for a query from the Index
5Node. Chunk of a Document with metadata (text, hash, source)
6Document. Raw file or string loaded from a data source
LlamaIndex models a RAG pipeline as a small set of named stages that flow from raw data to an answer. Knowing the stages and their order is what makes the framework's API make sense.
It starts with Document, the wrapper for a raw piece of content. A PDF page, an HTML page, a string. A Document is too coarse to retrieve against, so it gets split into Nodes. A Node is a chunk of the Document plus metadata: the original text, a content hash, the source document, and any extra fields you attach.
Nodes go into an Index. The Index is the queryable structure. The most common is a vector index where each Node has an embedding, but tree indexes, keyword indexes, and knowledge-graph indexes exist too.
At query time, a Retriever takes a question and returns the most relevant Nodes from the Index. A ResponseSynthesizer takes those Nodes plus the question and composes the final answer. Usually by calling an LLM with the Nodes as context.
A QueryEngine bundles the Retriever and ResponseSynthesizer behind one `query(question)` method. That is the surface callers actually use.
LlamaIndex's mental model is data-centric: every stage in the pipeline is a named transformation of structured units. The order goes ingestion (Document → Node → Index) then query (Retriever → ResponseSynthesizer), packaged for callers as a QueryEngine.
**Document** is the raw-content wrapper. A PDF page, an HTML page, a JSON record. All become Documents with a body and a metadata dict. Documents preserve the source identity so downstream stages can cite where an answer came from.
**Node** is the chunked, queryable unit. NodeParsers turn Documents into Nodes by applying chunking strategies. Sentence-window, token-count, semantic-similarity. Each Node carries the chunk text, a content hash for dedup, parent-document linkage for citation, and any custom metadata for filtering.
**Index** organizes Nodes into a queryable structure. The default `VectorStoreIndex` computes an embedding per Node and stores them in a vector backend. Other index types, KeywordTableIndex, TreeIndex, KnowledgeGraphIndex, exist for non-vector retrieval shapes.
**Retriever** is the query-time first half. It maps a question to a small set of relevant Nodes by similarity search, keyword match, or graph traversal depending on the index type.
**ResponseSynthesizer** is the query-time second half. It composes the final answer from the retrieved Nodes. Strategies include `compact` (single LLM call with Nodes concatenated), `tree_summarize` (hierarchical reduce), `refine` (sequential refinement), and `accumulate` (one answer per Node).
**QueryEngine** is the public-facing assembly: Retriever plus ResponseSynthesizer plus optional NodePostprocessors behind a single `query(question)` entry point.
Quick reference
TL;DR
LlamaIndex flows Document → Node → Index at ingest, then Retriever → ResponseSynthesizer at query, packaged as a QueryEngine for callers.
Picture turning a stack of books into a library you can ask questions of. You start with the books themselves. You cut each book into chapter-sized cards, each card stamped with its book and page number. You file the cards on shelves organized by topic so you can find them fast. When someone asks a question, a librarian pulls the few most relevant cards from the shelves and a writer turns those cards into a single short answer. The whole assembled service, librarian plus writer behind one counter, is what visitors actually use. They never see the cards, the shelves, or the books directly.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Short corpora where naive in-context loading beats retrieval on quality Below roughly a hundred pages it is often cheaper and better to dump everything in the context window
Agent-shaped workflows where the loop is the primary control structure LlamaIndex Workflows or LangGraph is a better fit; QueryEngine is a single-shot retrieve then synthesize, not an agent loop
Real-time streaming where the synthesis must start before retrieval finishes The standard pipeline is serial retrieve then synth; specialized streaming patterns need a different topology
Quick scan · 2–4 min
Everything important, quickly.
5 minutes: the six stages in order, the swap points, and the design choices each stage captures.
Code example
python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.node_parser import SentenceWindowNodeParser
from llama_index.core.response_synthesizers import get_response_synthesizer
# Doc -> Nodes
docs = SimpleDirectoryReader('./data').load_data()
parser = SentenceWindowNodeParser.from_defaults(window_size=3)
nodes = parser.get_nodes_from_documents(docs)
# Nodes -> Index
index = VectorStoreIndex(nodes)
# Retriever + Synthesizer -> QueryEngine
retriever = index.as_retriever(similarity_top_k=5)
synth = get_response_synthesizer(response_mode='tree_summarize')
query_engine = index.as_query_engine(retriever=retriever, response_synthesizer=synth)
response = query_engine.query('What did the postmortem identify as the root cause?')
Real-world examples
Real products, models, and research that use this idea.
1LlamaIndex 0.13 ships the canonical Document → Node → Index → QueryEngine pipeline as the default RAG flow.
2LlamaParse converts PDFs into Documents with preserved structure before NodeParser chunking, used heavily in enterprise document RAG.
3Cohere Rerank attaches as a NodePostprocessor between Retriever and ResponseSynthesizer in many production pipelines.
4Notion-style internal search systems frequently use LlamaIndex's auto-merging retriever to balance chunk-level recall with passage-level coherence.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you wire Cohere Rerank into this pipeline without touching the Retriever or Synthesizer?
Sign in or go Pro to see the approach for this follow-up.
QWhat happens to citation propagation if you replace VectorStoreIndex with TreeIndex?
Sign in or go Pro to see the approach for this follow-up.
QHow do you handle incremental ingest when only a few documents have changed?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Trying to query a Document directly without going through Nodes and an Index, then wondering why retrieval quality is poor.