Regularization

In LoRA, where in the forward pass does `lora_dropout` apply its mask?

Train loss drops while eval loss climbs at step 1200: name it and the standard fixes

Where inside the attention sub-layer is dropout typically applied during training, and what does dropping there teach the model?

Injecting noise into pre-softmax attention scores, goal and risk?