An engineer insists you need LangChain to do RAG. Refute that claim concretely. Describe the minimum viable RAG pipeline and then say when LangChain (or LlamaIndex) does actually earn its keep on top of it.
Free · 2 AI evals / day
RAG, retrieval-augmented generation, is a pattern, not a framework. The pattern is four steps:
1. **Embed** the user's question with an embedding model (`text-embedding-3-small`, `bge-large`, etc.).
2. **Retrieve** the top-k most similar chunks from a vector store (Pinecone, pgvector, Qdrant, Weaviate, Chroma).
3. **Format** the retrieved chunks into a prompt with the question.
4. **Complete** by calling a chat completion API and returning the response.
Written directly, this is about 80 lines of Python. No framework required. You can build it with the OpenAI SDK plus your vector store's client library. It works.
LangChain (or LlamaIndex) becomes valuable when the pipeline grows past that simple shape. Common triggers:
- **Hybrid retrieval.** Combine vector similarity with BM25 keyword search, then rerank with a cross-encoder. Each piece is a few lines; orchestrating the combination by hand becomes verbose.
- **Multi-index routing.** Some queries should hit a vector index, others a SQL database, others a graph index. A router that picks the right one per query is messy to hand-roll.
- **Response synthesizer strategies.** When the retrieved context is too long for one model call, you need refine, tree-summarize, or accumulate patterns. LlamaIndex packages these.
- **RAG embedded in an agent.** When RAG is one tool a larger agent uses, the agent framework handles the orchestration.
For the simple case the framework is overhead. For the complex case hand-rolling becomes more expensive than the framework tax. The misconception comes from confusing 'a popular tool exists for X' with 'X requires that tool'.
RAG is a four-step pattern: embed the query, retrieve nearest neighbors from a vector store, format the chunks into a prompt, call the model. Done directly with the OpenAI SDK and a vector store client (Pinecone, pgvector, Qdrant, Weaviate), this is roughly 80 lines of Python. No framework required to make RAG work.
LangChain and LlamaIndex earn their keep when composition complexity grows past the simple shape. The common triggers are:
**Hybrid retrieval.** Production RAG typically combines vector similarity with BM25 keyword search and a cross-encoder reranker. Each piece is a few lines, but orchestrating the combination by hand becomes verbose. LangChain's `EnsembleRetriever` and LlamaIndex's `QueryFusionRetriever` package this cleanly.
**Multi-index routing.** A real product often has questions that should hit different backends. Vector for QA, SQL for counts, summary for overview. A router that picks per-query is reasonable to hand-roll once and painful to extend. LlamaIndex's `RouterQueryEngine` and LangChain's `RunnableBranch` make this maintainable.
**Response synthesizer strategies.** When retrieved context exceeds the model's effective window, you need refine (iterate over chunks, building up the answer), tree-summarize (recursively summarize chunk groups), or accumulate (collect per-chunk answers and merge). LlamaIndex packages these as `ResponseSynthesizer` strategies.
**RAG as one step in an agent.** When RAG is a tool the agent uses among many, the agent framework (LangGraph, LangChain agents) handles the orchestration, observability, and persistence.
The right framing: frameworks are insurance against future complexity. Pay the premium when you expect to need it. For an MVP serving a single use case, the 80-line direct version is faster to build and easier to debug. The 'you need LangChain' framing confuses framework popularity with framework requirement.
Quick reference
TL;DR
Minimal RAG is four steps (embed, retrieve, format, complete) in 80 lines with no framework.
Picture a kid claiming you need a kitchen to make a sandwich. You point out that bread, peanut butter, and a knife do the job. They are right that the kitchen helps once you want to make ten different sandwiches with toasted bread and a side soup. But the kitchen is not the sandwich. RAG is the same. The 'kitchen' is the framework. The 'sandwich' is the four steps. Confusing the two leads teams to import five megabytes of code to do something the standard library could do in a screen of code.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Single vector store retrieval with no hybrid search and no routing The four-step pipeline in 80 lines is faster to build, easier to debug, and easier to read than the framework-wrapped equivalent
Hot-path latency budgets where every import and abstraction layer matters Framework import weight and per-call overhead are real; direct SDK calls minimize the stack between user code and the model
Teams without bandwidth to learn and operate framework abstractions The framework tax includes ramp time and ongoing version-drift maintenance; small teams without bandwidth pay more in framework friction than they save in code
Edge or embedded deployments with binary-size constraints LangChain pulls dozens of dependencies; embedded or serverless cold-start environments often cannot afford the import footprint
Quick scan · 2–4 min
Everything important, quickly.
4 minutes: state the four-step minimal pipeline, name a vector store concretely, list the complexity classes that flip the framework calculus, and frame the production answer as 'piecemeal adoption when buying something with the tax'.
Code example
python
# Minimum viable RAG: no framework, ~30 lines
from openai import OpenAI
import pinecone
client = OpenAI()
index = pinecone.Index('my-index')
def rag(question: str, k: int = 5) -> str:
# 1. Embed
q_vec = client.embeddings.create(
model='text-embedding-3-small', input=question
).data[0].embedding
# 2. Retrieve
matches = index.query(vector=q_vec, top_k=k, include_metadata=True).matches
chunks = [m.metadata['text'] for m in matches]
# 3. Format
context = '\n\n'.join(f'[{i+1}] {c}' for i, c in enumerate(chunks))
prompt = f'Answer using ONLY the context. If unsure, say so.\n\nContext:\n{context}\n\nQuestion: {question}'
# 4. Complete
resp = client.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': prompt}],
)
return resp.choices[0].message.content
print(rag('What is the company refund policy?'))
Real-world examples
Real products, models, and research that use this idea.
1Perplexity AI's RAG-style search backend is built directly on model APIs and a custom retrieval stack, not on LangChain
2Many enterprise search rebuilds in 2025-2026 use OpenAI or Anthropic SDKs plus pgvector and stay framework-free
3LlamaIndex's QueryFusionRetriever is a common pickup once teams need hybrid retrieval with reranking
4LangGraph is frequently adopted specifically for RAG-as agent tool patterns where the orchestration outgrows direct code
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QAt what point in a project's lifecycle does adopting a framework become net-positive?
Sign in or go Pro to see the approach for this follow-up.
QWhy might LlamaIndex be preferable to LangChain specifically for RAG workloads?
Sign in or go Pro to see the approach for this follow-up.
QWhat does 'eject from LangChain' look like operationally, and why does it matter for tool choice?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Mistaking framework popularity for framework requirement. The framework being widely used does not mean the underlying technique depends on it.