What is LLM-as-judge and why did it emerge as an evaluation approach?

**LLM-as-judge** means using one capable language model to grade the outputs of another model. You give the judge a rubric ("Is this answer helpful? Accurate? On topic?") and it returns a score or picks the better of two answers. It emerged to solve a real problem. Human evaluation is the gold standard, but it is slow and expensive. You cannot pay people to read millions of chatbot responses every week. The older alternative was string-match metrics like BLEU or ROUGE. Those compare the output to a reference answer word by word. They break for open-ended text because there are many correct ways to phrase the same answer. An LLM judge fixes both problems. It reads for meaning, so it recognises a correct answer phrased differently. It scales to millions of queries cheaply. And it applies the same rubric consistently across very different response types. The tradeoff: a judge is not a perfect oracle. It has measurable biases and needs occasional checking against real human labels. But for scalable evaluation, it is the dominant pattern in 2026.

**LLM-as-judge** uses a capable model to score or rank another model's outputs against a rubric. You prompt the judge with the candidate answer plus criteria, and it returns a score or a preference. It emerged from a gap. Human evaluation is accurate but slow and costly, so it cannot cover millions of production queries. String-match metrics like BLEU and ROUGE are cheap but brittle. They reward word overlap, not meaning, so a correct paraphrase scores poorly. The judge sits between these. It reads for semantic quality, recognises valid paraphrases, and applies a multi-dimensional rubric consistently. It scales to huge volumes at a fraction of human cost. The honest caveat is simple. A judge is a measurement instrument with known error. It has position bias and self-preference bias, and it is weaker on facts than on style. So you treat its score as a useful proxy, not as ground truth. The production pattern follows from that. Use the judge for scale, pin temperature to zero for stability, and periodically calibrate it against a small set of human labels.