Learning Rate

Why do large runs ramp learning rate before entering cosine decay?

Pick the most likely cause when training loss drops for 1k steps, plateaus for 2k, then drops again at step 3k

Loss spikes the moment warmup ends: which hyperparameter is the prime suspect?

Match training regime to sensible peak LR range

Fill in typical peak LRs for full FT, LoRA, and DPO

LoRA+: why does B want a higher learning rate than A?

Which of these reduce catastrophic forgetting in practice?