LoRA and QLoRA dominate the headlines in 2026, yet a sober production team would still pick full fine-tuning in specific situations. Make the case: when does full FT genuinely beat LoRA, and what is the actual quality gap?
Free · 2 AI evals / day
**Full fine-tuning** updates every weight in the model. **LoRA** trains a small low-rank correction and leaves the base weights frozen. LoRA is cheaper in nearly every dimension: less GPU memory, smaller checkpoints, fast to swap.
So why would anyone still pick the expensive option? Three conditions push the decision toward full fine-tuning.
First, lots of data. If you have 10,000 high-quality examples or more, the extra capacity of full fine-tuning is actually used. With small datasets the extra capacity becomes an overfitting vector.
Second, big distribution shift. If your domain is far from the base model's pretraining mix, like legal, biomedical, or a non-English script, every weight needs to move a little. A low-rank correction only changes the model along a few directions, which is fine for style nudges but cramped for a genuine domain shift.
Third, simple serving. LoRA's selling point at inference is adapter portability, the ability to swap several adapters against one frozen base. If you serve a single product or a single tenant, you never use that feature, so full fine-tuning's lack of it costs you nothing.
The quality gap on hard benchmarks is usually small, a few points, and DoRA narrows it further. But on the right workload, that small gap is worth the extra cost.
**Full fine-tuning still wins** in a narrow but real slice of production. Three conditions, each independent, push the decision in that direction.
The first is dataset size. With ten thousand or more high-quality examples, the extra capacity of full fine-tuning is actually used. Below that, the extra weights become an overfit vector and LoRA's regularising low-rank constraint is a feature, not a limit.
The second is distribution shift. Vanilla LoRA expresses the update as a sum of low-rank corrections, which is enough for instruction-style polish on a base model that already knows your domain. When the target distribution sits far from the base, like Japanese legal contracts, biomedical literature, or a low-resource language, the update needs to touch every weight and the low-rank constraint becomes a ceiling. Full fine-tuning takes that ceiling off.
The third is serving topology. LoRA's headline serving win is adapter portability. One frozen base, dozens of adapters swapped per request, kilobytes per skill. If you run a single tenant or a single product, you never use that feature, and the LoRA-versus-full trade collapses to pure quality.
The measured quality gap is usually small, a couple of points on MMLU-medical, LegalBench, or domain-specific reasoning tasks. DoRA narrows it further by giving LoRA a separate magnitude knob, and the strongest 2026 variants land within a point of full fine-tuning. But on a large, far-from-base, single-tenant workload, that small gap is worth the cost.
Quick reference
TL;DR
Full fine-tuning still wins when the dataset is large, the distribution shift from base is broad, and the serving stack is single-tenant. The gap on hard benchmarks is small but real.
Imagine a master chef trained on global cuisine, and you want them to cook only Japanese kaiseki. If you have just a few recipes to teach them, sticking a small notebook of tips in their pocket works fine: that is the popular cheap method. But if you have thousands of recipes and the cuisine is genuinely different from anything they trained on, you actually want them to relearn their habits across every dish, not just glance at a notebook. The full retraining is slower and more expensive, and it only pays off when you have lots of teaching material, the new style is genuinely far from the old, and you serve only this one restaurant. In every other case the notebook is a better deal.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Dataset is small (under a few thousand examples) Full fine-tuning's extra capacity overfits; LoRA's low-rank constraint acts as regularisation.
You need to serve many tenants or many skills against one frozen base Full fine-tuning produces one dense model per task; LoRA adapters hot-swap at kilobytes per skill.
Target distribution is close to base (mild instruction polish or format change) A low-rank correction is sufficient and far cheaper, so the full-FT premium pays for nothing.
You lack the GPUs or budget for a full optimizer state on the base model size Full fine-tuning of a 70B class model requires multi-GPU sharding; LoRA or QLoRA may fit on a single node.
Quick scan · 2–4 min
Everything important, quickly.
5 min: the three conditions for full FT, the capacity argument behind low-rank ceilings, the adapter-portability trade, where the measured gap appears, how DoRA narrows it, and the production decision framework.
Side-by-side
Axis
Full fine-tuning
LoRA / DoRA
Training memory
16+ bytes per param with Adam
Low single-digit GB for adapters
Quality ceiling
Highest on far-from-base domains
Within ~1 to 4 pts on most benchmarks
Checkpoint size
Full base weights per fine-tune
Megabytes per adapter
Adapter portability
None, one model per task
Many adapters per frozen base
Best regime
Large data, big shift, single tenant
Small data, near-base shift, multi-tenant serving
Overfit risk on small data
High, extra capacity hurts
Low, low-rank constraint regularises
Real-world examples
Real products, models, and research that use this idea.
1BloombergGPT trained a 50B model fully on financial corpora because the distribution shift from general web text was large enough to justify the cost.
2Med-PaLM 2 used full fine-tuning over medical literature where domain accuracy was the product, not a polish step.
3Mistral Large 3 customer deployments at law firms tend to choose full fine-tuning on tens of thousands of contract examples for single-tenant legal assistants.
4Llama 4 Maverick instruction-tuned releases now ship LoRA-tuned variants as the open community default, while frontier closed-tenancy deployments still run full fine-tunes when data warrants.
5DeepSeek V4 community fine-tunes use DoRA for low-resource languages where vanilla LoRA closes most but not all of the gap to full fine-tuning.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QOn a 10k-example legal contract dataset, how would you decide whether LoRA r=64 plus DoRA is enough or you need full fine-tuning?
Sign in or go Pro to see the approach for this follow-up.
QWhy does the quality gap show up more on reasoning and dense factual recall than on instruction following?
Sign in or go Pro to see the approach for this follow-up.
QIf your serving stack hosts twenty per-customer skills today and you fine-tune fully, what changes in your inference architecture?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Declaring full fine-tuning obsolete in 2026 because LoRA is cheaper. It is cheaper everywhere, but not always close enough in quality on large datasets that sit far from the base model to ignore the gap.