Put the steps of a basic RAG query pipeline in order.
- 1Embed the user query
- 2Generate the answer with the LLM
- 3Optionally rerank the retrieved chunks
- 4Search the vector index for top-k similar chunks
- 5Assemble the LLM prompt with retrieved context
Embed the query, retrieve top-k chunks, optionally rerank, stuff retrieved context into the prompt, then generate. Five steps, always in that order.
Imagine you ask a friend a tricky factual question. First, your friend writes down the key idea of your question on an index card (embed the query). Then they walk into a library, find the shelf with notes that match (vector search for top-k chunks), maybe filter to the most relevant few (rerank). Next they bring those notes back to the desk and lay them next to your question (assemble the prompt). Finally, they read everything and write an answer for you (generate). RAG is just that workflow, automated. The model is the friend, the embeddings are the index, the vector store is the library, and the LLM does the final writing.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
RAG (Retrieval-Augmented Generation) has become the default pattern for grounding LLM answers in fresh, private, or domain specific knowledge. The five step query pipeline is the universal skeleton, every production system layers extras on top of it, but the core ordering is fixed.
Understanding why the order is fixed (not just memorizing the steps) is what separates a candidate who can debug a broken RAG stack from one who can only describe it. The ordering falls out of data dependencies between stages, not convention.
This deep dive walks through each step, explains why the dependencies force the order, and shows where real-world stacks add complexity without breaking the skeleton.
Step 1: Embed the query
Turn the user's query into a vector using an embedding model. The same model that was used to embed the document chunks at index time. This is non-negotiable.
Why the same model: an embedding model places semantic content in a particular geometric space. Two different models produce vectors in two different spaces, comparing them with cosine similarity gives numbers but the numbers don't mean anything. A query embedded with text-embedding-3-small and searched against an index built with bge-large-en-v1.5 will return results, but ranking quality will be effectively random.
In practice this is one API call (text-embedding-3-small, bge-large-en-v1.5, cohere-embed-v3, etc.) and takes 20 to 100 milliseconds. Output dimension is typically 384, 768, 1024, or 1536 floats.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Perplexity runs query embed plus web retrieval plus a rerank plus Claude or GPT generation, the canonical web-RAG flow.
- GitHub Copilot Chat embeds the active query, retrieves repo chunks, and stuffs them into the prompt before generation.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhere would you add query rewriting in this pipeline?
Before step 1, so the rewritten query gets embedded. Useful when multi-turn questions reference earlier context the embedding can't see.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Putting reranking before retrieval. You can't rerank what you haven't retrieved, the reranker scores a candidate list.
60 second bullets to scan on the way to the call.
The five steps in order
Why embedding has to come before retrieval
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.