MMLU appears on every model leaderboard. Describe what it tests and its biggest blind spot.

**MMLU** stands for Massive Multitask Language Understanding. It is a benchmark of 14,042 multiple-choice questions covering 57 academic subjects, from abstract algebra and astronomy to world religions and US history. Each question offers four options, and the model picks one. What MMLU measures well is knowledge breadth. If a model scores 85% on MMLU, it can recognize the correct answer across a wide range of topics. This is useful because it tells you the model has been exposed to diverse training data and can recall it accurately. The biggest blind spot is the gap between recognition and generation. Picking the correct option from four choices does not prove the model can explain the concept clearly, handle a follow-up question, or produce a well-structured answer when no options are given. A model that scores 90% on MMLU might still write confusing, verbose, or unhelpful free-form responses because the benchmark never tests those skills.

**MMLU** (Massive Multitask Language Understanding) is a benchmark of 14,042 four-choice questions across 57 academic subjects. It measures knowledge breadth: how well the model can identify the correct answer from a fixed set of options across domains ranging from STEM to humanities. The blind spot is fundamental to the format. Multiple-choice recognition is not open-ended generation. A model can score highly by matching patterns between the question and the correct option without understanding the underlying concept deeply enough to explain it, debate it, or apply it to a novel situation. This is why models with near-identical MMLU scores can produce dramatically different quality free-form responses. A second blind spot is benchmark contamination. Because MMLU is a fixed, publicly available question set, models that have seen the exact questions during training can score higher than their actual knowledge warrants. MMLU-Pro was introduced as a harder successor with ten-choice questions and reasoning requirements, but even that cannot fully solve the recognition versus generation gap.