Every regime here updates a model, but they touch different parts and carry different risk, so each wants a different **peak learning rate**.
Full fine-tuning of a 7B model moves all the weights at once. Those weights already encode useful knowledge, so you step gently: roughly 1e-5 to 5e-5, with a short warmup and cosine decay.
LoRA SFT freezes the base and trains tiny adapter matrices that start from zero. Nothing precious is at risk, and the adapters need a bigger push to learn, so you go about 10 times higher, near 1e-4 to 5e-4. QLoRA on a 70B base sits in the same range because the 4-bit frozen weights do not care about your learning rate.
DPO learns from preference pairs, and its gradient comes from a tiny difference between two log-probabilities. That signal is fragile, so you drop way down to 5e-7 to 5e-6.
Continued pretraining keeps teaching the full model new text. To avoid wiping out what it already knew, you stay near 1e-5, close to where the original schedule ended.
Peak learning rate scales inversely with how fragile the update is and how much existing knowledge sits at risk. That single idea orders all five regimes.
Full fine-tuning of a 7B model moves every weight, so a hot rate causes forgetting or divergence. The safe band is 1e-5 to 5e-5 with cosine decay and about 3 percent warmup. LoRA SFT freezes the base and trains zero-initialised adapters, so it tolerates roughly 10 times more: 1e-4 to 5e-4. QLoRA on a 70B base shares that band, since the 4-bit frozen weights are inert to the learning rate. You still tune the adapters in full precision.
DPO is the outlier on the low end. Its gradient flows from a small log-ratio between policy and reference, which destabilises easily, so 5e-7 to 5e-6 is standard. Continued pretraining stays near 1e-5, close to the tail of the original cosine schedule, to limit catastrophic forgetting.
The through-line is simple. Frozen-base adapter methods run hot, while full-parameter and preference methods run cold.
Quick reference
TL;DR
Peak LR tracks how fragile the update is: LoRA SFT runs hottest near 1e-4 to 5e-4, full fine-tuning sits mid, and DPO plus continued pretraining run coldest.
Picture how bold each change is when you fix up something you already care about. If you scribble on a fresh sticky note that started blank, you can press hard and write fast, since there is nothing to ruin. If you touch up a finished painting, you dab gently so you do not smear years of work. If you nudge someone toward a very subtle taste, the hint is so faint that any big stroke wipes it out, so you barely tap. And if you keep teaching a wise old friend brand-new books, you whisper the lessons so they do not forget everything they already knew. The rule is the same everywhere: the more precious and finished the thing, the smaller your step should be.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Picking a learning rate purely from the model parameter count Size correlates weakly with the safe band; the frozen versus live distinction matters far more than scale.
Reusing one favourite learning rate across SFT, DPO, and continued pretraining These regimes span four orders of magnitude, so a single value will overshoot or stall at least three of them.
Treating the quoted bands as exact targets rather than starting points Optimal peak LR shifts with batch size, dataset size, and adapter rank, so a short sweep still beats any default.
Applying these SFT and DPO bands to online RLHF with PPO Online RL has its own actor and critic dynamics, so the supervised and preference bands do not transfer cleanly.
Quick scan · 2–4 min
Everything important, quickly.
5 min: build the fragility framing for peak LR, then place the five bands in order, then explain why LoRA and QLoRA match, why DPO and continued pretraining run coldest, and how warmup plus decay protect live weights.
Side-by-side
Regime
Peak LR band
Why this band
LoRA r=16 SFT
1e-4 to 5e-4
Frozen base, zero-init adapters tolerate ~10x higher LR
QLoRA on 70B
1e-4 to 2e-4
Same as LoRA; 4-bit base is frozen and LR-inert
Full FT of 7B
1e-5 to 5e-5
All weights live; hot LR forgets or diverges
Continued pretraining
around 1e-5
Near schedule floor to avoid catastrophic forgetting
DPO from SFT checkpoint
5e-7 to 5e-6
Fragile log-ratio gradient destabilises easily
Real-world examples
Real products, models, and research that use this idea.
1Hugging Face TRL ships default DPO configs near 5e-7 and LoRA SFT configs near 2e-4, mirroring this fragility ordering.
2Axolotl and Unsloth recipes for Llama 4 and Qwen 3 LoRA fine-tunes default to 1e-4 to 2e-4 peak LR with cosine decay.
3The QLoRA reference setup keeps the 4-bit base frozen and tunes adapters near 2e-4, matching standard LoRA despite the large base.
4Meta and Mistral continued-pretraining runs resume near the cosine-schedule floor around 1e-5 to limit catastrophic forgetting.
5Zephyr and Tulu preference-tuning recipes set DPO peak LR in the 5e-7 to 5e-6 band, well below their SFT phase.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does a zero-initialised LoRA adapter tolerate a much higher peak learning rate than full fine-tuning?
Sign in or go Pro to see the approach for this follow-up.
QWhy is the DPO learning rate roughly 100 times lower than the SFT rate on the same checkpoint?
Sign in or go Pro to see the approach for this follow-up.
QHow would you adjust peak LR when moving from LoRA r=16 to r=256?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Reusing one favourite learning rate across regimes. A LoRA-tuned 2e-4 will blow up full fine-tuning and is roughly 100 times too hot for DPO.