Temperature

Walk through how top_p (nucleus) sampling truncates the distribution

Compare top_k and top_p as truncation strategies for sampling

How does the temperature parameter reshape the sampling distribution?

Dividing pre-softmax attention scores by an extra factor > 1 at inference does what?

Pick what happens to attention weights when pre-softmax scores are divided by a temperature T > 1

Attention temperature divides QK^T; sampling temperature divides output logits, distinguishwhere each lives.

Flashcard: what does the temperature parameter do during LLM generation and how should you set it?

When does temperature > 0 produce better results than temperature = 0 in production LLM applications?

Why does self-consistency require temperature > 0 and what specifically happens if you set T = 0?

Explain why self-consistency requires temperature > 0 and quantify the gain it produces over single pass CoT on math benchmarks.

Spot the error in this explanation of temperature.