Where inside the attention sub-layer is dropout typically applied during training, and what does dropping there teach the model?

Short answer·Medium·4.0 · 0·~3 min·Asked atRoblox Robust Intelligence Zed·Relevant atAi4bharat Cerebras Meta Replicate

Attempt it

Where inside the attention sub-layer is dropout typically applied during training, and what does dropping there teach the model? Mention the dominant placement and any secondary placements.

Free · 2 AI evals / day

Attention dropout sits in a very specific place: right after the softmax that produces the attention weight matrix. The attention output is computed in three steps: 1. Raw scores: `S = Q K^T / sqrt(d_k)`. 2. Weights: `A = softmax(S)`, each row isa probability distribution over keys. 3. Output: `out = A V`. Dropout is applied to A in step 2 before step 3. Each entry of A is independently set to zero with probability p, and the surviving entries are scaled up by `1 / (1 - p)` so the expected sum stays roughly the same. What does dropping there teach? It discourages **single-key collapse**. If a query has learned to pour all its weight on one specific key, dropout will sometimes zero that key out and force the query to recover by spreading weight across other keys. Over training this regularizes the attention pattern to be more robust. There is also a secondary dropout slot after the output projection `W_O`, but that one is just generic residual dropout, not attention-specific. Most modern LLM pretraining (Llama, Mistral, Qwen) actually uses zero attention dropout because the data scale alone is enough regularization.

The dominant attention dropout slot is on the **post-softmax weight matrix** `A = softmax(Q K^T / sqrt(d_k))`. Each entry is zeroed independently with probability p, and the surviving entries are scaled by `1 / (1 - p)` to preserve the expected weighted sum. The semantic effect is what makes this placement special. Dropping individual attention weights forces each query to **spread its mass** across multiple keys instead of collapsing onto a single sharp peak. If the model has learned a brittle 'always look at token 7' pattern, dropout breaks that pattern often enough that gradient descent has to find a more distributed solution. It is regularization specifically against over-confident, narrow attention. A secondary slot sits **after the output projection** `W_O`, on the residual-stream activations leaving the sub-layer. That one is generic residual dropout, not attention-specific, and it has the same role here as it does anywhere else in the network. The two slots use independent dropout rates in most reference implementations. Vaswani's original transformer used 0.1 in both. Modern frontier LLMs (Llama, Mistral, Qwen, DeepSeek) typically set attention dropout to 0 at pretraining scale, the data islarge enough that explicit regularization stops being the bottleneck, but supervised fine-tuning recipes still use it because the data is smaller and overfitting is easier.

TL;DR

Attention dropout is applied to the post-softmax weight matrix to discourage single-key collapse; a second slot lives after the output projection as residual dropout.

Memory aid

Easy to grasp

Imagine a student doing a group project who only ever asks one teammate for help. When the test comes, that teammate is sick and the student panics. A teacher who randomly tells the student 'pretend that teammate is unavailable today, work with someone else' is forcing the student to learn to ask many teammates. That is what attention dropout does. It randomly silences some of the attention links during training so the model has to spread its bets and not get addicted to one particular key. At test time everyone is available, but the student has learned not to depend on any single source.

Key concepts

Dropout Regularization Softmax

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Dropout placement in the attention sub-layer is one of those topics that sounds settled until you actually look at five different reference implementations and see four different patterns. The Vaswani 2017 paper specified one canonical slot; PyTorch's MultiheadAttention adopted it; modern large-scale pretraining quietly removed it; LoRA fine-tuning brought it back. Understanding what each placement does and why the convention has drifted is the difference between guessing at hyperparameters and choosing them intentionally.

This deep dive walks the canonical placement, explains why the post-softmax slot is the structurally correct one, surveys the secondary slots and the variants, and ends with a practical decision guide for when to use attention dropout in 2026.

The canonical placement: post-softmax on the weight matrix

The attention sub-layer produces three meaningful tensors during the forward pass:

The raw score matrix S = Q K^T / sqrt(d_k).
The weight matrix A = softmax(S) (each row is a probability distribution over keys).
The output out = A V, which is then run through W_O.

Attention dropout is applied between steps 2 and 3, directly on A.

The mechanics

Each entry A[i, j] is independently set to zero with probability p. The surviving entries are scaled up by 1 / (1 - p) to preserve the expected weighted-sum magnitude. In PyTorch:

python

A = F.softmax(scores, dim=-1)
A = F.dropout(A, p=p, training=True)  # zeros and rescales
out = A @ V

Why post-softmax and not pre-softmax

Dropout pre-softmax sets some raw scores to zero. After softmax, those zero entries still get exponentiated to exp(0) = 1, so they are not truly zeroed in the output. The intended regularization (preventing the model from depending on one specific key) is diluted.

Dropout post-softmax sets actual weights to zero. The model directly experiences 'sometimes the key I want to read from is unavailable', which is the regularization signal we want.

The rule of thumb: regularize where the network would otherwise be brittle. The brittle dependency is on the final attention weight, not on the raw score.

Why this regularization matters

The secondary slot: residual dropout after W_O

Variants and alternatives

The 2026 trend: turning attention dropout off

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Vaswani 2017 original Transformer: dropout 0.1 on the post-softmax weight matrix and 0.1 residual dropout after each sub-layer.
Llama 4 Maverick, DeepSeek V4, Mistral Large 2 pretraining: attention dropout typically set to 0.0 because data scale provides implicit regularization.

What an interviewer would ask next. Try answering before peeking at the approach.

QIf you wanted to regularize attention without using dropout, what alternatives exist?

Entropy regularization on the attention distribution (penalize low-entropy weight rows to discourage collapse), DropHead (whole-head dropping), LayerDrop (whole-block dropping), and label smoothing on the output. Each targets a different failure mode.

1 more follow-up an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Applying dropout on Q, K, or V before the dot product instead of on the post-softmax weights. The standard attention-specific slot is on A, not on the inputs.

60 second bullets to scan on the way to the call.

Position of attention dropout in the QKV pipeline
Why the 1 over (1 - p) rescaling step is necessary

Primary sources. Browse if you want the original framing.