Sampling

Walk through how top_p (nucleus) sampling truncates the distribution

Compare top_k and top_p as truncation strategies for sampling

How does the temperature parameter reshape the sampling distribution?

Pair each decoding strategy with its determinism and diversity profile.

Walk through the shape of an OpenAI compatible chat completions request and response.

Attention temperature divides QK^T; sampling temperature divides output logits, distinguishwhere each lives.

Flashcard: what does the temperature parameter do during LLM generation and how should you set it?

Flashcard: what is self-consistency prompting and what does it produce as a final answer?

Compare greedy, sampling and beam search at inference: cost and best use case.