Which statements correctly describe the benefits and limitations of a panel of judges eval setup?

A **judge panel** uses several LLM judges instead of one, then aggregates their verdicts by averaging scores or taking a majority vote. The idea is the same as asking a committee instead of one reviewer: errors that differ between members tend to cancel out. Three of the options are correct. A panel built from different model families (Claude, GPT, Gemini) averages out family-specific quirks, because each family's stylistic preferences point in different directions. Requiring agreement across three or more judges also smooths out random one-off mistakes, lowering variance on ambiguous examples. And running several judges per query naturally costs more, while giving a more stable aggregate signal. Three options are wrong because they overstate the guarantee. A panel reduces position bias and self-preference bias, but does not eliminate them. If every judge prefers the first answer it sees, averaging cannot cancel a bias they all share. Using judges from the same family gives you redundancy but not diversity, so the systematic family bias survives the average. And full agreement never proves correctness: if all judges hallucinate in the same direction, the consensus is confidently wrong.

A **judge panel** aggregates several LLM judges (average, majority vote, or weighted consensus) instead of trusting one. The correct options describe what that actually buys you. Diversity across families averages out family-specific bias. A GPT judge over-rewards GPT-style answers; a Claude judge over-rewards Claude-style ones. Pointed in different directions, those biases partly cancel in the mean. Consensus across three or more judges also cuts variance on ambiguous examples, because one judge's stochastic misread gets outvoted. And more judge calls per query cost more, while producing a steadier aggregate. The wrong options overstate the guarantee. Averaging only cancels bias that is uncorrelated across judges. Position bias is shared by almost every LLM family. If all judges prefer the first option, the panel mean still prefers the first option. Reduction, not elimination. Same-family judges give redundancy without diversity, so the systematic family bias survives untouched. And consensus is not correctness. If every judge hallucinates the same wrong fact, they agree, and the verdict is confidently wrong. The distinction that matters is uncorrelated error, which panels reduce, versus shared bias, which panels cannot.