Pass At K

Evaluating a code generation model on real-world tasks beyond HumanEval: which metrics cover correctness, efficiency, and style?

HumanEval is the go-to code generation benchmark. Describe the task it gives the model and how it decides if the answer is correct.

Explain the pass@k metric and why the unbiased estimator matters for code gen evaluation

What does pass@k measure and how is it computed to reduce variance?