Training Stability

QLoRA OOMs at step 7000 after one clean epoch on a 24 GB GPU: which non-obvious causes deserve a look?

With max_grad_norm=1.0 and a raw gradient L2 of 12.5, what is the post-clip norm and scale factor?

Why ramp the learning rate during warmup instead of starting at the peak value?

Gradient norm clipping, name the failure mode it exists to prevent

A DPO run spikes to NaN around step 400 after a clean start: explain the most likely cause

Why does normalizing Q and K before the dot product help training at scale?

Pre-norm versus post-norm: which placement makes deep stacks stable?

Why is LoRA's B initialised to zero and A to Gaussian. What breaks if both are Gaussian?

What specifically goes wrong if you remove the √d_k scaling from attention?