Walk through LlamaIndex's Document → Node → Index → QueryEngine pipeline and explain why this shape slots into RAG more naturally than LangChain's Runnable model
Walk an interviewer through LlamaIndex's Document → Node → Index → QueryEngine pipeline. Explain what each primitive owns, and argue why this data-shaped worldview lets a from-scratch RAG pipeline ship faster than building the same thing on top of LangChain Runnables.
Free · 2 AI evals / day
LlamaIndex's RAG pipeline has four named stages, and each stage corresponds to a real noun in the RAG workflow.
**Document.** The raw input. A PDF, a webpage, a markdown file, a database row. A `Document` object carries the text plus metadata (source path, author, date). It is what comes into the system.
**Node.** A chunk derived from a Document. The ingestion pipeline splits Documents into Nodes. Fixed-size text chunks, semantically split chunks, or sentence-level chunks depending on the splitter you pick. Each Node has its own metadata (parent Document ID, position, hash) and is the unit of retrieval.
**Index.** The data structure that organises Nodes for fast lookup. The `VectorStoreIndex` embeds Nodes and stores the embeddings; the `KeywordTableIndex` builds an inverted term map; the `TreeIndex` builds a hierarchical summary. You pick the index based on the retrieval shape you need.
**QueryEngine.** The callable that takes a user query and returns an answer. Internally it pairs a `Retriever` (pulls candidate Nodes from the Index) with a `ResponseSynthesizer` (composes the final answer from the retrieved Nodes).
The equivalent in LangChain is to build a chain like `query | retriever | format | model | parser` from LCEL Runnables. It works, but you are assembling RAG semantics from generic plumbing. LlamaIndex names those concepts directly.
LlamaIndex's four-stage pipeline is named for the RAG nouns themselves, which is what makes the framework feel productive on RAG-shaped problems.
**Document.** Raw input plus metadata. Loaders (`SimpleDirectoryReader`, vendor connectors via LlamaHub) produce them.
**Node.** A retrievable chunk. `NodeParser` implementations (`SimpleNodeParser`, `SentenceSplitter`, `SemanticSplitterNodeParser`) decide the splitting strategy. Each Node carries its own metadata, position, and embedding handle.
**Index.** The organising structure. `VectorStoreIndex` (the default) embeds Nodes and stores them in a vector database; `SummaryIndex` keeps all Nodes for sequential reads; `KeywordTableIndex` builds an inverted term map; `KnowledgeGraphIndex` extracts entities and relations. Each index type implements the same interface, so swapping retrieval shapes is a one-line change.
**QueryEngine.** The packaged callable. It bundles a `Retriever` (gives candidate Nodes for a query), an optional list of `NodePostprocessors` (re-ranking, similarity filtering, recency weighting), and a `ResponseSynthesizer` (refine, tree-summarize, accumulate, simple). One call site, one configurable object.
The productivity argument over LangChain is not that LangChain cannot do RAG. It can. It is that the default path is shorter in LlamaIndex because the primitives are already RAG-shaped. Response synthesizer strategies, hybrid retrievers, node postprocessors, and metadata filtering are first-class concepts with documented modes. On LangChain you assemble equivalent behaviour by composing generic Runnables, which means more code, more decisions, and a longer ramp.
Quick reference
TL;DR
LlamaIndex names the RAG nouns directly, Document, Node, Index, QueryEngine, so the default path to a working RAG pipeline is shorter than assembling the same shape from LangChain's generic Runnables.
Imagine two kitchens. The first has labelled stations: prep bench (Document), cutting board (Node), pantry (Index), and pass window (QueryEngine). Every recipe in the cookbook tells you exactly which station does what. The second kitchen has only generic counters labelled 'work surface 1' through 'work surface 5,' and the same recipes ask you to assign each step to a surface yourself. Both kitchens can produce the same meal. The labelled one gets dinner on the table faster because the design of the room matches the design of the recipe.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Non-RAG workloads. Agents, tool-using assistants, multi-step reasoning chains LlamaIndex's primitives are RAG-shaped; for orchestration-shaped problems, LangChain or LangGraph is the better default
Production pipelines that need durable checkpointing and HITL interrupts Wrap the LlamaIndex QueryEngine as a tool inside LangGraph; that combination gets you both the RAG primitives and the orchestration durability
Tiny corpora (under a few hundred chunks) that fit in one context The full Document/Node/Index/QueryEngine ceremony is unjustified; pass the corpus directly in the prompt
Workflows where the retrieval logic is bespoke and does not map onto Index / Retriever abstractions Forcing LlamaIndex's shape onto a non-fit creates abstraction friction; reach for raw vector-store SDK calls instead
Quick scan · 2–4 min
Everything important, quickly.
6 minutes: the four stages, what each owns, the response-synthesizer mode landscape, NodePostprocessors, and when to mix LlamaIndex retrieval with LangChain orchestration.
Code example
python
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
from llama_index.core.node_parser import SentenceSplitter
from llama_index.core.postprocessor import SimilarityPostprocessor
from llama_index.core.response_synthesizers import ResponseMode
# Document
docs = SimpleDirectoryReader("./corpus").load_data()
# Node
parser = SentenceSplitter(chunk_size=512, chunk_overlap=64)
nodes = parser.get_nodes_from_documents(docs)
# Index
index = VectorStoreIndex(nodes)
# QueryEngine with a postprocessor and a synthesizer mode
qe = index.as_query_engine(
similarity_top_k=8,
node_postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.7)],
response_mode=ResponseMode.TREE_SUMMARIZE,
)
print(qe.query("What are the GenAI OpenTelemetry conventions?"))
Side-by-side
RAG stage
LlamaIndex primitive
LangChain equivalent
Raw input
Document + DocumentLoader
Document + DocumentLoader
Chunking
NodeParser → Node
TextSplitter → Document chunks
Organising
Index (Vector / Summary / Keyword / KG)
VectorStore + Retriever
Retrieval
Retriever + NodePostprocessor
Retriever + (ad-hoc filter Runnables)
Answer composition
ResponseSynthesizer with named modes
LCEL chain you assemble
Packaged call site
QueryEngine
Custom Runnable chain
Real-world examples
Real products, models, and research that use this idea.
1LlamaIndex's `chat_with_data` and `query_engine_tools` quickstarts ship working RAG pipelines in roughly 20 lines of Python. The default-path argument made concrete.
2Production RAG systems at companies like Notion AI and Mendable rely on LlamaIndex's NodePostprocessor + ResponseSynthesizer composition for their retrieval cores.
3LlamaHub hosts hundreds of `DocumentLoader` integrations (Notion, Slack, Confluence, GitHub, Discord) that produce Documents directly, skipping bespoke ingestion code.
4LlamaIndex's `QueryFusionRetriever` is the published-pattern way to do RRF-blended hybrid retrieval; LangChain's `EnsembleRetriever` is the rough equivalent in the LangChain ecosystem.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhen would you pick `tree_summarize` over `refine` as your response-synthesizer mode?
Sign in or go Pro to see the approach for this follow-up.
QHow would you build a hybrid (vector + BM25) retrieval pipeline in LlamaIndex?
Sign in or go Pro to see the approach for this follow-up.
QHow do you connect a LlamaIndex QueryEngine into a LangGraph agent?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Calling LlamaIndex 'just a wrapper over LangChain' and missing that the Document/Node/Index/QueryEngine vocabulary is itself the productivity win for RAG-shaped problems.