Swe Bench

Design a diagnostic eval for an agentic workflow with 62% task success across 5 tools and 10+ steps that reveals WHY tasks fail, not just whether they pass.

Evaluating a code generation model on real-world tasks beyond HumanEval: which metrics cover correctness, efficiency, and style?

How should you score a multi-step agent where final answer accuracy is 15% but most steps are correct?

Walk through SWE-bench, AgentBench, GAIA, and TAU-bench: what they measure, their shared blind spots, and how to interpret agent benchmark numbers.

What structural property of SWE-bench makes it a harder eval than HumanEval for code agents?

What does SWE-bench measure that AgentBench does not, and why are both needed?

Match each agent benchmark to what it primarily measures

How does a coding agent's scope differ from inline code completion?