Training Stability

QLoRA OOMs at step 7000 after one clean epoch on a 24 GB GPU: which non-obvious causes deserve a look?

With max_grad_norm=1.0 and a raw gradient L2 of 12.5, what is the post-clip norm and scale factor?

Why ramp the learning rate during warmup instead of starting at the peak value?

Gradient norm clipping, name the failure mode it exists to prevent

A DPO run spikes to NaN around step 400 after a clean start: explain the most likely cause

PremiumWhy does normalizing Q…

Pre-norm versus post-norm: which placement makes deep stacks stable?

Why is LoRA's B initialised to zero and A to Gaussian. What breaks if both are Gaussian?

PremiumWhat specifically goes wrong…