You tell the model 'respond in exactly 3 bullet points' and it gives you 5. Which benchmark specifically targets this class of failure?

The benchmark is **IFEval** (Instruction Following Evaluation). It was built to test one specific thing: does the model do what you explicitly asked it to do in terms of format? IFEval works by giving the model prompts with verifiable constraints. Respond in exactly 3 bullet points. Use fewer than 50 words. Include the word 'therefore' at least twice. Do not use the letter 'e'. Each constraint is something a simple program can check without understanding the content. This is different from benchmarks like MMLU (tests knowledge), MT-Bench (tests multi turn conversation quality), or HumanEval (tests code generation). None of those isolate whether the model followed an explicit formatting instruction. The practical value of IFEval is that it catches a class of failure that rubric based evaluation misses. A judge model rating answer quality might give 5/5 to a brilliant five bullet answer when you asked for three. IFEval catches that the model ignored your constraint, which matters in production when you need structured, predictable output.

**IFEval** (Instruction Following Evaluation) specifically targets this class of failure. It uses programmatically verifiable formatting constraints: bullet point counts, word count limits, language constraints, keyword inclusion and exclusion, and structural requirements like 'respond as a JSON object.' The key design property is deterministic scoring. Every constraint is checkable by a simple program, so there is no judge model variance. This complements open ended quality evals (MT-Bench, Chatbot Arena) by isolating a dimension they overlook: simple compliance with explicit instructions. MMlu tests factual knowledge across domains. MT-Bench tests multi turn conversation quality with a judge model. HumanEval tests code generation against unit tests. None of these isolate format adherence as a first class evaluation dimension. IFEval fills that gap and has become a standard component of model evaluation suites since its introduction.