Retrieval-Augmented Generation

Design a fine-tune that teaches a model to refuse when the retrieved context lacks the answer.

You have 500KB of relevant documentation that exceeds the 200K-token context window of your production model. How should you choose between context stuffing (fit what you can) and RAG retrieval?

Flashcard: what is a vector database, and why does RAG need one?

Flashcard: what is similarity search in RAG, and how does it differ from keyword search?

Flashcard: what does 'grounding' mean in the context of RAG, and why is it the whole point?

Flashcard: what is an embedding in a RAG pipeline, and what role does it play?

Flashcard: what does chunking mean in RAG, and why do we chunk at all?

Your team is comparing a vanilla LLM chatbot against a RAG chatbot. What is the single most important difference between them at query time?

Your team is tuning top-k for the retriever in a production RAG system. What is the strongest reason NOT to just push top-k to its highest possible value?

Design a production RAG system serving 10,000 concurrent users over a 100M-document corpus, with a P95 latency target of 2 seconds. Walk through the architecture and the bottleneck at each layer.

What is the core mechanism that distinguishes Self-RAG (Asai et al. 2023) and Corrective-RAG (CRAG, 2024) from a standard single-shot retrieve then generate pipeline?

Fill in the blanks: query expansion / query rewriting in RAG and what it costs.

Predict the retrieval trace: how a multi-hop RAG agent answers a chained question that single-shot RAG would fail on.

Why does hybrid retrieval (BM25 plus dense vectors) typically outperform either alone in production RAG?

Describe a concrete, production runnable mechanism to detect hallucinations in a RAG answer (claims unsupported by the retrieved chunks), as the answer is generated or just after.

Design a golden eval set for a production RAG system serving a legal research product. Explain how you build it, how big it should be, what each example contains, and what you measure with it.

Match each embedding-model selection scenario to the model attribute or check that most determines the right choice.

Your production RAG costs $1M/month. The CFO wants this cut in half with a max 1-point faithfulness regression. What highest leverage cost optimizations do you deploy, in priority order?

Spot the bug: a junior engineer's conversational RAG implementation produces nonsense retrievals on turn 3+. Find the error in the retrieval logic.

Flashcard: what role does the LLM's context window play in a RAG pipeline?

Select all techniques that meaningfully improve the reliability of source citations in a production RAG answer (chunks resolving back to real source documents).

You are tuning chunk size in a RAG pipeline. What is the core tradeoff between very small chunks (e.g. 128 tokens) and very large chunks (e.g. 2000 tokens)?

Select all components that are required to build a minimal end to end RAG system (the absolute basics, not nice to haves like rerankers or eval harnesses).

Three concrete situations where fine-tuning is the wrong tool

Which of these are valid reasons NOT to fine-tune?