RAG System Design

When an LLM in a RAG pipeline behaves differently because of the retrieved context, what is actually changing about the model?

Your RAG system's faithfulness metric just dropped from 0.91 to 0.74 while context recall stayed flat at 0.85. What's the most likely pipeline layer to investigate first?

Complete the claim: the two-axis decomposition of RAG eval that diagnoses which layer failed.

Order the layers you'd investigate when debugging a RAG faithfulness regression that just shipped to production, from most likely to least likely cause.

Why do cross-encoder rerankers outperform dual-encoder retrievers at picking the most relevant chunks, given both models can be trained on the same data?

Which of the following are augmentation sub-decisions in a production RAG pipeline? Select all that apply.

Flashcard: name the three augmentation sub-decisions in a production RAG pipeline.

Analyze MMLU's strengths and failure modes as an LLM benchmark

Design a cost-efficient LLM-as-judge eval strategy for a system processing 1M queries/day

How do you set up an A/B test for an LLM system change, and what's different from standard A/B testing?

Design a RAG evaluation harness, what do you measure and how?