Order the stages of a LlamaIndex RAG pipeline from raw data to a query response

Order steps·Medium·4.0 · 0·~1 min·Asked atGnani Krutrim Tcs

Attempt it

1QueryEngine. The assembled retrieve then synthesize entrypoint
2Index. Nodes organized into a queryable structure (vector / keyword / tree)
3ResponseSynthesizer. Composes the final answer from retrieved Nodes
4Retriever. Pulls candidate Nodes for a query from the Index
5Node. Chunk of a Document with metadata (text, hash, source)
6Document. Raw file or string loaded from a data source

LlamaIndex models a RAG pipeline as a small set of named stages that flow from raw data to an answer. Knowing the stages and their order is what makes the framework's API make sense. It starts with Document, the wrapper for a raw piece of content. A PDF page, an HTML page, a string. A Document is too coarse to retrieve against, so it gets split into Nodes. A Node is a chunk of the Document plus metadata: the original text, a content hash, the source document, and any extra fields you attach. Nodes go into an Index. The Index is the queryable structure. The most common is a vector index where each Node has an embedding, but tree indexes, keyword indexes, and knowledge-graph indexes exist too. At query time, a Retriever takes a question and returns the most relevant Nodes from the Index. A ResponseSynthesizer takes those Nodes plus the question and composes the final answer. Usually by calling an LLM with the Nodes as context. A QueryEngine bundles the Retriever and ResponseSynthesizer behind one `query(question)` method. That is the surface callers actually use.

LlamaIndex's mental model is data-centric: every stage in the pipeline is a named transformation of structured units. The order goes ingestion (Document → Node → Index) then query (Retriever → ResponseSynthesizer), packaged for callers as a QueryEngine. **Document** is the raw-content wrapper. A PDF page, an HTML page, a JSON record. All become Documents with a body and a metadata dict. Documents preserve the source identity so downstream stages can cite where an answer came from. **Node** is the chunked, queryable unit. NodeParsers turn Documents into Nodes by applying chunking strategies. Sentence-window, token-count, semantic-similarity. Each Node carries the chunk text, a content hash for dedup, parent-document linkage for citation, and any custom metadata for filtering. **Index** organizes Nodes into a queryable structure. The default `VectorStoreIndex` computes an embedding per Node and stores them in a vector backend. Other index types, KeywordTableIndex, TreeIndex, KnowledgeGraphIndex, exist for non-vector retrieval shapes. **Retriever** is the query-time first half. It maps a question to a small set of relevant Nodes by similarity search, keyword match, or graph traversal depending on the index type. **ResponseSynthesizer** is the query-time second half. It composes the final answer from the retrieved Nodes. Strategies include `compact` (single LLM call with Nodes concatenated), `tree_summarize` (hierarchical reduce), `refine` (sequential refinement), and `accumulate` (one answer per Node). **QueryEngine** is the public-facing assembly: Retriever plus ResponseSynthesizer plus optional NodePostprocessors behind a single `query(question)` entry point.

TL;DR

LlamaIndex flows Document → Node → Index at ingest, then Retriever → ResponseSynthesizer at query, packaged as a QueryEngine for callers.

Memory aid

Easy to grasp

Picture turning a stack of books into a library you can ask questions of. You start with the books themselves. You cut each book into chapter-sized cards, each card stamped with its book and page number. You file the cards on shelves organized by topic so you can find them fast. When someone asks a question, a librarian pulls the few most relevant cards from the shelves and a writer turns those cards into a single short answer. The whole assembled service, librarian plus writer behind one counter, is what visitors actually use. They never see the cards, the shelves, or the books directly.

Key concepts

LLM Frameworks RAG Embeddings

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

LlamaIndex looks intimidating from the outside because the API has so many class names. The trick to learning it quickly is to see that those classes are a small ordered pipeline, not a sprawling library. Knowing the six stages and their boundaries makes the entire framework readable.

This deep dive walks each stage in order, says what input it takes and what output it produces, and names the design choice the stage is asking you to make.

Phase one: ingestion. Document, Node, Index

The ingestion phase turns raw data into a queryable structure. It runs once per data load or on a schedule; the result is the persistent index that query-time stages will read.

Document

A Document is the raw-content wrapper. It carries a text body, a metadata dict, and a source identifier. Loaders produce Documents from filesystems, APIs, databases, and SaaS sources via the LlamaHub ecosystem.

The Document layer is where structural information is preserved. If a PDF has page numbers, the Documents carry them in metadata. If an HTML page has a canonical URL, it travels with the Document. This is what later lets citations point back to the right place.

Node

Nodes are the queryable units. A NodeParser takes Documents and splits them into Nodes according to a chunking strategy. Choices include:

SentenceWindowNodeParser. Keeps a wide context window per Node while indexing on a narrower retrieval window. Good for long-form QA where the retrieved chunk needs context.
TokenTextSplitter. Fixed token-count chunks with overlap. The cheap default.
SemanticSplitter. Splits at embedding-distance changes. Higher quality, slower.
HierarchicalNodeParser. Produces parent and child Nodes for auto-merging retrieval.

Each Node carries the chunk text, a content hash, parent-document references, and a metadata dict inherited from the Document plus any Node-specific additions. The content hash is the dedup key when you re-ingest.

Index

The Index organizes Nodes into a queryable structure. VectorStoreIndex is the default: each Node gets an embedding, embeddings live in a vector backend (Pinecone, Weaviate, Postgres pgvector, in-memory).

Alternative Index types address non-vector query shapes:

KeywordTableIndex. Inverted index style keyword retrieval for exact-match workloads.
TreeIndex. Recursive summary tree; useful for very large corpora where summary-first then drill-down is the right query pattern.
KnowledgeGraphIndex. Entity-relation graph; useful for structured-query questions over relational corpora.

The Index type is the highest-impact ingestion choice. Picking the wrong one, vector when the query is keyword-shaped, or vector when the corpus has heavy structural relationships, costs more downstream than any later tuning.

Phase two: query. Retriever, NodePostprocessor, ResponseSynthesizer

The QueryEngine and the swappability payoff

Situations where this technique stops working.

2–4 min · Everything important, quickly.

python

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.node_parser import SentenceWindowNodeParser
from llama_index.core.response_synthesizers import get_response_synthesizer

# Doc -> Nodes
docs = SimpleDirectoryReader('./data').load_data()
parser = SentenceWindowNodeParser.from_defaults(window_size=3)
nodes = parser.get_nodes_from_documents(docs)

# Nodes -> Index
index = VectorStoreIndex(nodes)

# Retriever + Synthesizer -> QueryEngine
retriever = index.as_retriever(similarity_top_k=5)
synth = get_response_synthesizer(response_mode='tree_summarize')
query_engine = index.as_query_engine(retriever=retriever, response_synthesizer=synth)

response = query_engine.query('What did the postmortem identify as the root cause?')

Real products, models, and research that use this idea.

LlamaIndex 0.13 ships the canonical Document → Node → Index → QueryEngine pipeline as the default RAG flow.
LlamaParse converts PDFs into Documents with preserved structure before NodeParser chunking, used heavily in enterprise document RAG.

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you wire Cohere Rerank into this pipeline without touching the Retriever or Synthesizer?

Add a CohereRerank as a NodePostprocessor on the QueryEngine; the retriever still does first-stage selection, the postprocessor reranks before the synthesizer sees the Nodes.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Trying to query a Document directly without going through Nodes and an Index, then wondering why retrieval quality is poor.

60 second bullets to scan on the way to the call.

The six ordered stages of a LlamaIndex RAG pipeline
What a Document carries versus what a Node carries

Primary sources. Browse if you want the original framing.