Scale AI

Which metric best measures whether a RAG answer is grounded in the retrieved context?

Design a RAG evaluation harness, what do you measure and how?

Which of these are valid concerns when using LLM-as-judge for evaluation?

Work out the KV cache reduction when 32 query heads share 8 KV heads

A consumer facing LLM product is about to launch. Describe the safety evaluation that should run first.

Automated eval vs human eval: which is better for catching subtle factual errors in medical text?

You tell the model 'respond in exactly 3 bullet points' and it gives you 5. Which benchmark specifically targets this class of failure?

How do you evaluate a customer service chatbot that handles multi-turn conversations? What dimensions matter?

The PM asks how to tell if the chatbot got better after the last deploy. Show setting up an A/B test.

An MCP tools/call result comes back. Which content types could it contain?