LLM As Judge Basics

Define LLM-as-judge and explain the one problem it was invented to solve.

Tradeoffs between pairwise LLM-judge comparisons and scalar scoring rubrics

What is the primary scalability limitation of pairwise LLM-judge evaluation compared to scalar scoring?

What is LLM-as-judge and why did it emerge as an evaluation approach?

How does a chain-of-thought judge differ from a direct-score judge, and when is each better?

What is the primary advantage of chain-of-thought (CoT) judge prompts over direct-score prompts?

Which strategies reduce LLM-as-judge evaluation cost at scale without sacrificing reliability?