Walk through how top_p (nucleus) sampling truncates the distribution

**top_p** sampling, also called nucleus sampling, is a way to throw out the long tail of unlikely tokens before picking the next word. The procedure runs in four steps. First, sort the post-softmax probabilities in descending order so the most likely token sits at the front. Second, walk down the sorted list and accumulate probability mass until the running total first crosses the threshold `p` (commonly 0.9). Third, discard every token after that prefix; the prefix is called the nucleus. Fourth, renormalize the survivors so their probabilities sum to 1, then sample one. The key feature is that the nucleus size adapts to the distribution shape. When the model is very confident, two or three tokens cover 90 percent of the mass and the nucleus is small. When the model is uncertain (think open creative writing or ambiguous user input), the nucleus opens up to include more candidates. Modern chat APIs like GPT-5.5 and Claude Sonnet 4.6 expose `top_p` as a primary sampling knob, usually defaulting to 1.0 (no truncation) with temperature doing the shaping.

**top_p** (nucleus) sampling is an adaptive cutoff applied to the post-softmax distribution. The recipe is: sort probabilities descending, take the smallest prefix whose cumulative mass first exceeds `p`, discard the tail, renormalize, and sample. The adaptivity is the whole point. On a peaked step where the top token holds probability 0.95, the nucleus at `p = 0.9` contains exactly one token; sampling becomes deterministic for that step. On a flat step where the top ten tokens are all near 0.05, the nucleus expands to include all ten or more. The cutoff tracks the model's per-step confidence rather than imposing a fixed cardinality. That is the structural advantage over top_k. A fixed K either admits garbage tail tokens when the model is confident (K is too large for the peak) or chops off plausible alternatives when the model is uncertain (K is too small for the spread). top_p sidesteps both failure modes by letting the set size float. In practice, chat models pair top_p with temperature: temperature reshapes the softmax (sharper or flatter), then top_p truncates the reshaped distribution. OpenAI, Anthropic, Gemini, and every open-weights serving stack expose both. The default for chat is usually `temperature=1, top_p=1`, with creative endpoints lowering top_p to 0.9 or 0.8 to keep outputs on-topic.