Perplexity
Also known as: PPL
Exp(average cross-entropy) on held-out text; lower means the model is less surprised by real data.
An intrinsic evaluation metric for language models: the exponential of the average negative log-likelihood per token. Lower perplexity means the model assigns higher probability to held-out text: a better generative fit.
In practice
Standard metric for pre-training quality. Interviews probe its weakness (poor proxy for instruction-following) and why it's not used for chat eval.
How it compares
Perplexity is one specific automated metric; LLM evaluation is the umbrella practice spanning many.
Related topics
Related terms
Hallucination
When a model confidently makes up something that isn't true.
LLM Evaluation
Measuring whether an LLM does what you want, beyond "looks fine to me".
Reasoning Model
An LLM trained to reason at length internally before answering. Slower and more expensive, but much better on hard problems.
Guardrails
Pre- and post-processing layers that block bad inputs/outputs and enforce policy on top of an LLM.
MMLU (Massive Multitask Language Understanding)
Multiple-choice benchmark across 57 academic subjects; the standard "raw knowledge" headline number.
HumanEval
Code-generation benchmark: 164 problems with hidden unit tests, scored by whether the generated code passes.