Amazon

Faithfulness scores are high, yet users keep calling the answers unhelpful — what is the eval missing?

A new enterprise customer onboards with an empty index and zero eval data — stand up their RAG and prove it works

Your new reranker looks better offline — prove it actually improves retrieval in production before full rollout

Which metrics isolate the retriever's quality, separate from how good the final answer reads?

Match each retriever-evaluation metric to what it actually measures

Scoring RAG faithfulness with an LLM judge and no gold answers — how it works and where it goes wrong

Walk through SWE-bench, AgentBench, GAIA, and TAU-bench: what they measure, their shared blind spots, and how to interpret agent benchmark numbers.

Flashcard: what is the ReAct prompting pattern?

You have 500KB of relevant documentation that exceeds the 200K-token context window of your production model. How should you choose between context stuffing (fit what you can) and RAG retrieval?

Three concrete situations where fine-tuning is the wrong tool