RAG System Design

Flashcard: what does 'grounding' mean in the context of RAG, and why is it the whole point?

Your team is comparing a vanilla LLM chatbot against a RAG chatbot. What is the single most important difference between them at query time?

Your team is tuning top-k for the retriever in a production RAG system. What is the strongest reason NOT to just push top-k to its highest possible value?

Design a production RAG system serving 10,000 concurrent users over a 100M-document corpus, with a P95 latency target of 2 seconds. Walk through the architecture and the bottleneck at each layer.

What is the core mechanism that distinguishes Self-RAG (Asai et al. 2023) and Corrective-RAG (CRAG, 2024) from a standard single-shot retrieve then generate pipeline?

Fill in the blanks: query expansion / query rewriting in RAG and what it costs.

Predict the retrieval trace: how a multi-hop RAG agent answers a chained question that single-shot RAG would fail on.

Why does hybrid retrieval (BM25 plus dense vectors) typically outperform either alone in production RAG?

Describe a concrete, production runnable mechanism to detect hallucinations in a RAG answer (claims unsupported by the retrieved chunks), as the answer is generated or just after.

Design a golden eval set for a production RAG system serving a legal research product. Explain how you build it, how big it should be, what each example contains, and what you measure with it.

Match each embedding-model selection scenario to the model attribute or check that most determines the right choice.

Your production RAG costs $1M/month. The CFO wants this cut in half with a max 1-point faithfulness regression. What highest leverage cost optimizations do you deploy, in priority order?

Spot the bug: a junior engineer's conversational RAG implementation produces nonsense retrievals on turn 3+. Find the error in the retrieval logic.

Flashcard: what role does the LLM's context window play in a RAG pipeline?

Select all techniques that meaningfully improve the reliability of source citations in a production RAG answer (chunks resolving back to real source documents).

You are tuning chunk size in a RAG pipeline. What is the core tradeoff between very small chunks (e.g. 128 tokens) and very large chunks (e.g. 2000 tokens)?

Select all components that are required to build a minimal end to end RAG system (the absolute basics, not nice to haves like rerankers or eval harnesses).

Multiple users complain that your AI assistant 'ignores the documents I uploaded and just makes things up'. Before involving an ML engineer, which pipeline layer should you investigate first?

Which of the following are recommended directives for a production RAG system prompt to maximize grounding? Select all that apply.

How would you detect that your production RAG vector index has gone stale, before users notice degraded answers?

Predict the RRF (Reciprocal Rank Fusion) output ranking given these two retrievers' results.

Match each RAGAS metric to what it specifically measures about a RAG pipeline.

Why do augmentation prompt edits cause disproportionately more production quality incidents in RAG systems than LLM model swaps?

Spot the augmentation config bug in this production RAG YAML that's causing 'lost in the middle' faithfulness regressions on long-context queries.

In a production RAG system with a 2-second end to end latency budget for non-streaming responses, which step typically dominates and where should the optimization effort go?