Name the additive bias term in attention and the two things it most commonly carries.

The additive bias term in attention is a pre-softmax matrix `B` added to the scaled dot product `QK^T / sqrt(d_k)`. The full canonical formula with bias is: $$\text{scores} = \frac{QK^\top}{\sqrt{d_k}} + B, \quad \text{weights} = \text{softmax}(\text{scores})$$ The same B slot gets reused for two completely different jobs. **Job 1: Masking.** Set entries of B to `-inf` for positions that should not contribute to attention. After softmax those positions get weight 0. The two common masks are: - Causal mask: `B[i, j] = -inf` for j > i, so a query at position i cannot attend to future positions. - Padding mask: `B[i, j] = -inf` for any j that corresponds to a padding token. **Job 2: Positional bias.** ALiBi sets `B[i, j] = -m_h * |i - j|` per head, adding a linear distance penalty that gives the model a sense of position without an input-side positional embedding. These two uses are independent, you can use both at once. Modern serving stacks usually compose a causal mask with whatever positional bias scheme the model uses.

The attention **bias matrix** sits at the pre-softmax score level. The canonical form is: $$\text{scores} = \frac{QK^\top}{\sqrt{d_k}} + B$$ Then softmax runs row-wise to produce attention weights, then those weights multiply V to produce the context vector. B is the only place in standard attention where an additive per-position signal can be injected without changing the rest of the math. The two dominant uses share that one slot. **Masking** uses `-inf` entries. The softmax of `-inf` is exactly 0, so any masked position contributes nothing to the weighted sum. Causal masks block future positions during autoregressive generation; padding masks block padding tokens during batched inference; document attention masks block cross-document attention during packed pretraining. **Positional bias** uses finite negative values. ALiBi sets `B[i, j] = -m_h * |i - j|` where m_h is a fixed per-head slope. The penalty grows with distance, giving the model a relative position signal that is well-defined at any distance (which is why ALiBi extrapolates to longer than training sequences). T5's relative position embeddings use a similar idea with learned per-bucket biases. The two uses compose freely. A typical Llama-style attention layer applies a causal mask (`-inf` for future positions) plus, in ALiBi-style models, a per-head linear distance bias. The other options in the question (bias on V, inside the dot product, learned scalar) describe interventions at the wrong stage; none of them correspond to the standard attention bias slot.