Injecting noise into pre-softmax attention scores, goal and risk?

**Attention noise** is a training-time trick that adds small random perturbations to the pre-softmax attention scores. The goal is regularization, not speed. Without any noise, individual attention heads can collapse onto a single key. The softmax over scores produces a near one-hot distribution where almost all the weight goes to one token and the rest get essentially zero. This sounds great (the head 'knows what to look at') but it usually means the model has overfit to a shortcut: it memorized that key X always wins for query Y on training data. Adding a small amount of Gaussian noise to scores before softmax flattens the distribution slightly. The head still prefers its top keys, but it has to consider alternatives every step. This pushes the model toward smoother, more robust attention patterns that generalize better. The risk is the same knob in reverse. Too much noise drives attention toward near-uniform mixing. The head loses its ability to focus, useful signal is destroyed, and the model trains poorly. The technique is treated like dropout, small coefficient, often decayed across training.

**Attention noise** is regularization applied to the pre-softmax scores inside an attention layer. The goal is to keep individual heads from collapsing into overconfident low-entropy distributions. A naive training run can produce heads where one query routes to a single key with weight near 1.0. The head looks crisp but is brittle, it has learned a hard shortcut on the training distribution and breaks on slight perturbations at inference. The fix is to add a noise term `epsilon ~ N(0, sigma^2)` to each pre-softmax score: $$s_{ij} \leftarrow s_{ij} + \epsilon_{ij}, \quad \epsilon_{ij} \sim \mathcal{N}(0, \sigma^2)$$ The noise slightly flattens softmax. The head still prefers its top candidates but is forced during training to keep some weight on alternatives. Over many batches this acts like an entropy regularizer on the attention distribution. The risk lives in the same parameter. Push sigma too high and the noise overwhelms the actual scores; softmax flattens toward uniform; the head effectively averages over all keys with no focus. Useful routing signal is destroyed, perplexity stalls, and the model fails to learn the long-range dependencies attention was supposed to find. It is treated operationally like attention dropout, in fact, attention dropout (zeroing entries of the attention weight matrix) is a closely related regularizer at the post-softmax stage. Both target the same failure mode of over-peaked attention.