Attention temperature divides QK^T; sampling temperature divides output logits, distinguishwhere each lives.

A transformer doing token generation runs through many softmaxes, not just one. **Attention temperature** and **sampling temperature** act on different softmaxes. Attention temperature divides the pre-softmax score inside every attention layer. After computing `scores = QK^T / sqrt(d_k)`, you optionally divide by an extra factor T_attn before softmax. Lower T_attn produces peakier attention weights over keys, the head focuses sharply on one or two keys. Higher T_attn flattens the weights, the head blends more keys into the context vector. Sampling temperature divides the very last logits over the vocabulary. After the model has run through all attention layers and produced output logits of shape (d_vocab,), you divide those logits by T_sample before sampling the next token. Lower T_sample makes sampling more greedy and deterministic. Higher T_sample makes sampling more random and creative. The two knobs are fully independent. You can run sharp attention with diverse sampling for a model that focuses hard on the prompt but generates creative outputs. You can also run flat attention with greedy sampling, though this is rarely useful. The canonical `1 / sqrt(d_k)` scaling inside attention is a fixed constant of the architecture, not the attention temperature. Attention temperature is an additional multiplier on top.

**Attention temperature** and **sampling temperature** live at different softmaxes in the generation pipeline. Attention temperature acts mid-layer. Inside every attention head, the score computation is: $$s_{ij} = \frac{q_i \cdot k_j}{\sqrt{d_k} \cdot T_\text{attn}}$$ Note that `1 / sqrt(d_k)` is a fixed architectural constant; T_attn is the extra knob. Lower T_attn produces peakier attention weights (sharper retrieval, single-key focus). Higher T_attn flattens weights toward uniform mixing. Sampling temperature acts at the head of the model, on the final vocabulary logits. After the residual stream is projected to d_vocab and produces logits `z`, sampling computes: $$p_\text{vocab} = \text{softmax}(z / T_\text{sample})$$ Then a token is sampled from p_vocab. Lower T_sample is greedy-leaning; higher T_sample is creative. The two knobs are mechanically independent and can coexist in a single forward pass. They have different downstream effects: - T_attn affects which keys each query attends to at every layer. Set it too high and the model loses the ability to focus, internal routing collapses to uniform mixing. - T_sample affects only the final discrete choice of token. Set it too high and outputs become random; set it to zero and you get greedy decoding. A common interview confusion is to treat T_attn as just another name for the 1 / sqrt(d_k) scaling. They are different: sqrt(d_k) is a fixed normalizer that keeps variance independent of head dimension, T_attn is an additional tunable multiplier. Many production stacks expose T_sample to users but never expose T_attn.