Order the stages of a LlamaIndex RAG pipeline from raw data to a query response

Order steps·Medium·4.0 · 0·~1 min·Asked atMicrosoft Databricks Perplexity

Attempt it

1QueryEngine. The assembled retrieve then synthesize entrypoint
2Index. Nodes organized into a queryable structure (vector / keyword / tree)
3ResponseSynthesizer. Composes the final answer from retrieved Nodes
4Retriever. Pulls candidate Nodes for a query from the Index
5Node. Chunk of a Document with metadata (text, hash, source)
6Document. Raw file or string loaded from a data source

LlamaIndex models a RAG pipeline as a small set of named stages that flow from raw data to an answer. Knowing the stages and their order is what makes the framework's API make sense. It starts with Document, the wrapper for a raw piece of content. A PDF page, an HTML page, a string. A Document is too coarse to retrieve against, so it gets split into Nodes. A Node is a chunk of the Document plus metadata: the original text, a content hash, the source document, and any extra fields you attach. Nodes go into an Index. The Index is the queryable structure. The most common is a vector index where each Node has an embedding, but tree indexes, keyword indexes, and knowledge-graph indexes exist too. At query time, a Retriever takes a question and returns the most relevant Nodes from the Index. A ResponseSynthesizer takes those Nodes plus the question and composes the final answer. Usually by calling an LLM with the Nodes as context. A QueryEngine bundles the Retriever and ResponseSynthesizer behind one `query(question)` method. That is the surface callers actually use.

LlamaIndex's mental model is data-centric: every stage in the pipeline is a named transformation of structured units. The order goes ingestion (Document → Node → Index) then query (Retriever → ResponseSynthesizer), packaged for callers as a QueryEngine. **Document** is the raw-content wrapper. A PDF page, an HTML page, a JSON record. All become Documents with a body and a metadata dict. Documents preserve the source identity so downstream stages can cite where an answer came from. **Node** is the chunked, queryable unit. NodeParsers turn Documents into Nodes by applying chunking strategies. Sentence-window, token-count, semantic-similarity. Each Node carries the chunk text, a content hash for dedup, parent-document linkage for citation, and any custom metadata for filtering. **Index** organizes Nodes into a queryable structure. The default `VectorStoreIndex` computes an embedding per Node and stores them in a vector backend. Other index types, KeywordTableIndex, TreeIndex, KnowledgeGraphIndex, exist for non-vector retrieval shapes. **Retriever** is the query-time first half. It maps a question to a small set of relevant Nodes by similarity search, keyword match, or graph traversal depending on the index type. **ResponseSynthesizer** is the query-time second half. It composes the final answer from the retrieved Nodes. Strategies include `compact` (single LLM call with Nodes concatenated), `tree_summarize` (hierarchical reduce), `refine` (sequential refinement), and `accumulate` (one answer per Node). **QueryEngine** is the public-facing assembly: Retriever plus ResponseSynthesizer plus optional NodePostprocessors behind a single `query(question)` entry point.

Quick reference

TL;DR

LlamaIndex flows Document → Node → Index at ingest, then Retriever → ResponseSynthesizer at query, packaged as a QueryEngine for callers.

Memory aid

Easy to grasp

Picture turning a stack of books into a library you can ask questions of. You start with the books themselves. You cut each book into chapter-sized cards, each card stamped with its book and page number. You file the cards on shelves organized by topic so you can find them fast. When someone asks a question, a librarian pulls the few most relevant cards from the shelves and a writer turns those cards into a single short answer. The whole assembled service, librarian plus writer behind one counter, is what visitors actually use. They never see the cards, the shelves, or the books directly.

Key concepts

LLM Frameworks RAG Embeddings

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 minutes: the six stages in order, the swap points, and the design choices each stage captures.

Code example

python

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.node_parser import SentenceWindowNodeParser
from llama_index.core.response_synthesizers import get_response_synthesizer

# Doc -> Nodes
docs = SimpleDirectoryReader('./data').load_data()
parser = SentenceWindowNodeParser.from_defaults(window_size=3)
nodes = parser.get_nodes_from_documents(docs)

# Nodes -> Index
index = VectorStoreIndex(nodes)

# Retriever + Synthesizer -> QueryEngine
retriever = index.as_retriever(similarity_top_k=5)
synth = get_response_synthesizer(response_mode='tree_summarize')
query_engine = index.as_query_engine(retriever=retriever, response_synthesizer=synth)

response = query_engine.query('What did the postmortem identify as the root cause?')

Real-world examples

Real products, models, and research that use this idea.

LlamaIndex 0.13 ships the canonical Document → Node → Index → QueryEngine pipeline as the default RAG flow.
LlamaParse converts PDFs into Documents with preserved structure before NodeParser chunking, used heavily in enterprise document RAG.
Cohere Rerank attaches as a NodePostprocessor between Retriever and ResponseSynthesizer in many production pipelines.
Notion-style internal search systems frequently use LlamaIndex's auto-merging retriever to balance chunk-level recall with passage-level coherence.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you wire Cohere Rerank into this pipeline without touching the Retriever or Synthesizer?

QWhat happens to citation propagation if you replace VectorStoreIndex with TreeIndex?

QHow do you handle incremental ingest when only a few documents have changed?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Trying to query a Document directly without going through Nodes and an Index, then wondering why retrieval quality is poor.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.