Match training regime to sensible peak LR range

Every regime here updates a model, but they touch different parts and carry different risk, so each wants a different **peak learning rate**. Full fine-tuning of a 7B model moves all the weights at once. Those weights already encode useful knowledge, so you step gently: roughly 1e-5 to 5e-5, with a short warmup and cosine decay. LoRA SFT freezes the base and trains tiny adapter matrices that start from zero. Nothing precious is at risk, and the adapters need a bigger push to learn, so you go about 10 times higher, near 1e-4 to 5e-4. QLoRA on a 70B base sits in the same range because the 4-bit frozen weights do not care about your learning rate. DPO learns from preference pairs, and its gradient comes from a tiny difference between two log-probabilities. That signal is fragile, so you drop way down to 5e-7 to 5e-6. Continued pretraining keeps teaching the full model new text. To avoid wiping out what it already knew, you stay near 1e-5, close to where the original schedule ended.

Peak learning rate scales inversely with how fragile the update is and how much existing knowledge sits at risk. That single idea orders all five regimes. Full fine-tuning of a 7B model moves every weight, so a hot rate causes forgetting or divergence. The safe band is 1e-5 to 5e-5 with cosine decay and about 3 percent warmup. LoRA SFT freezes the base and trains zero-initialised adapters, so it tolerates roughly 10 times more: 1e-4 to 5e-4. QLoRA on a 70B base shares that band, since the 4-bit frozen weights are inert to the learning rate. You still tune the adapters in full precision. DPO is the outlier on the low end. Its gradient flows from a small log-ratio between policy and reference, which destabilises easily, so 5e-7 to 5e-6 is standard. Continued pretraining stays near 1e-5, close to the tail of the original cosine schedule, to limit catastrophic forgetting. The through-line is simple. Frozen-base adapter methods run hot, while full-parameter and preference methods run cold.