A model card reports perplexity of 3.2. Explain what that number means in plain English.

**Perplexity** measures how well a language model predicts text it has not seen before. Think of it as a surprise score: the lower the perplexity, the less surprised the model is by the text. A perplexity of 3.2 has an intuitive meaning. At each position in the text, the model is effectively choosing among 3.2 equally likely next tokens. If perplexity were 1.0, the model would predict every next token perfectly. If perplexity were 50,000 (the vocabulary size), the model would be guessing randomly. Mathematically, perplexity is the exponential of the average cross-entropy loss across all tokens in the test text. Cross-entropy measures how far the model's predicted probability distribution is from the actual next token. Lower cross-entropy means the model's predictions are closer to reality, which means lower perplexity. The important caveat is that perplexity only measures language modeling quality. A model with low perplexity is good at predicting what natural text looks like. That does not mean it follows instructions, avoids hallucinations, or produces factually correct answers. Those are downstream task capabilities that require separate evaluation.

**Perplexity** quantifies a language model's surprise on held-out text. A perplexity of 3.2 means the model is, on average, as uncertain as if it were choosing uniformly among 3.2 equally likely next tokens at each step. Lower is better. The formal definition connects to information theory. Perplexity is the exponential of the average cross-entropy loss: $$\text{PPL} = \exp\!\left(-\frac{1}{N}\sum_{i=1}^{N} \log p(x_i \mid x_{<i})\right)$$ where N is the number of tokens, and `p(x_i | x_{<i})` is the probability the model assigns to the actual next token given the preceding context. When the model assigns high probability to every actual token, cross-entropy is low and perplexity approaches 1. When the model is uncertain, cross-entropy rises and perplexity climbs. Perplexity is an intrinsic metric: it measures the model on its own terms (how well it predicts text) rather than on an external task. This makes it useful for comparing language model architectures and training runs, but limited for evaluating whether the model is actually useful. The critical gap: low perplexity does not imply correctness. A model can assign high probability to fluent but factually wrong text because it has learned what plausible text looks like, not what is true. Instruction following, factual accuracy, and safety require extrinsic evaluation with task-specific metrics.