Log Probabilities
Also known as: Logprobs, Token probabilities
Per-token log probabilities from the model: confidence signal, eval input, and classification trick.
The log of the probability the model assigned to each generated token. Returned by most LLM APIs (logprobs flag). Useful for confidence scoring, eval, structured-output validation, and classification via constrained sampling.
In practice
Underrated production tool. Interviews probe using logprobs for confidence thresholds and as judges in evals.
Related topics
Related terms
KV Cache
Cache attention's K and V tensors per layer so each new token doesn't re-process every prior token.
Hallucination
When a model confidently makes up something that isn't true.
LLM Evaluation
Measuring whether an LLM does what you want, beyond "looks fine to me".
Temperature
Sampling knob (low = focused, high = diverse), applied to the logits before softmax.
Quantization
Run the model at lower numerical precision to save memory and accelerate inference.
FlashAttention
A memory-aware attention kernel that's 2-4x faster than vanilla, with identical math.