Softmax

PremiumOrder the tail end…

Does asking for top-k logprobs add real compute cost per token, or only bytes on the wire?

How does the temperature parameter reshape the sampling distribution?

At which step in the attention pipeline does the mask actually take effect?

Each row of the post-softmax attention weight matrix corresponds to which side of the QK product?

Dividing pre-softmax attention scores by an extra factor > 1 at inference does what?

Pick what happens to attention weights when pre-softmax scores are divided by a temperature T > 1

Along which axis of the QK^T score matrix is softmax applied inside attention?

Name the two properties softmax guarantees for every row of the attention weight matrix

Which token most often serves as an attention sink in pretrained autoregressive LLMs?

Why not just retrain models to eliminate attention sinks entirely?

How does softmax turn an attention score into an attention weight?

Name the scalar that QK^T is divided by inside scaled dot product attention.

Predict what softmax produces when every key in a row is masked out.

Where inside the attention sub-layer is dropout typically applied during training, and what does dropping there teach the model?

What does the dot product q . k actually measure inside an attention layer?

Identify the context vector inside scaled dot product attention and what produces it.

For a causal mask over a 4-token sequence, what value sits at row 2, column 3?

Attention temperature divides QK^T; sampling temperature divides output logits, distinguishwhere each lives.

Injecting noise into pre-softmax attention scores, goal and risk?

Your team debates 32K vs 128K vs 256K vocab. What is the core tradeoff they should frame?

PremiumWhat's the 'attention sink'…

Why is softmax used in attention rather than alternatives like sparsemax or simple sum normalization?

Complete the canonical attention formula: Attention(Q, K, V) = ___(QKᵀ / ___) · V

Where in attention is FP32 still used, and what breaks if you push everything to FP16/BF16?