Which statements about QLoRA are true?
QLoRA stacks four tricks: a 4-bit NF4 base, double quantisation of the constants, paged optimisers, and LoRA adapters kept in bf16.
Imagine shrinking a huge reference encyclopedia so it fits in your backpack. NF4 is a clever compression that keeps the most-used pages crispest, because most pages cluster around the middle. Double quantisation then compresses the little index cards that say how each section was shrunk. The encyclopedia itself you never write in; you keep frozen, compressed pages. Instead you carry a thin notebook of full-quality sticky notes (the adapters) where all your new learning happens. And if your desk runs out of room while studying, paged optimisers quietly move some scratch paper to a drawer and fetch it back later. Together these four tricks let you fine-tune a model that would normally need a server rack on a single gaming GPU.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
QLoRA is the recipe that made fine-tuning a 70-billion-parameter model on a single consumer-class GPU normal rather than heroic. It is not one idea but four, stacked so that each attacks a different slice of the memory budget. The multi-select format here is a precise probe: it rewards candidates who can name each component AND say exactly which tensor it touches, and it punishes the ones who memorised the headline 4-bit number without understanding the rest.
The single mental model that unlocks every option is a clean split. Quantise only what stays frozen. Keep full precision wherever a gradient must flow. The base model is frozen, so it gets crushed to 4-bit. The LoRA adapters are learnable, so they stay in bf16. Everything else in the recipe is either a storage trick on the frozen side or a runtime safety valve.
This deep dive walks all four real components, then dissects the two distractors that interviewers plant on purpose. Each distractor confuses two layers that look adjacent but are actually independent: where learning happens, and where activations live. A final section puts the numbers together so you can see why the recipe collapses a server-rack job onto one card.
NF4: why 4-bit, but not uniform 4-bit
The frozen base is stored in NF4, a 4-bit NormalFloat format. The key insight is that trained neural-net weights are not uniformly spread; they cluster tightly around zero in a roughly Gaussian shape. A naive uniform INT4 grid wastes resolution on the rare large-magnitude weights and starves the dense region near zero.
NF4 fixes this by deriving its sixteen quantisation levels from the quantiles of a unit Normal distribution. Buckets are packed densely where weight mass is high and spread out in the tails. Under a Gaussian prior on the weights, this minimises expected quantisation error at four bits. It is built to be the optimal four-bit data type for normally distributed data, which is what a layer of pretrained weights approximates after normalisation.
There is one practical wrinkle worth naming. NF4 is applied block-wise, with each block normalised by its own absolute-maximum constant before the Normal buckets are applied. That per-block normalisation keeps the Gaussian assumption locally valid even when different parts of the network have different weight scales. It is exactly the set of constants that the next trick goes on to compress.
This is why option A is correct and why describing NF4 as just INT4 with a fancy name is a red flag. The information-theoretic argument is the whole point: same four bits, more preserved signal, because the bucket spacing matches the data distribution rather than ignoring it.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| QLoRA component | What it touches | Precision / effect |
|---|---|---|
| NF4 base | Frozen base weights | 4-bit, Normal-distributed buckets |
| Double quantisation | Per-block scaling constants | Saves about 0.37 bits per parameter |
| Paged optimisers | Adam optimiser state | Spills to CPU on memory spikes |
| LoRA adapters | Learnable low-rank factors | Stay in bf16 for stable gradients |
| Gradient checkpointing | Activations (orthogonal) | Still used; saves activation memory |
Real products, models, and research that use this idea.
- Hugging Face PEFT plus bitsandbytes ships QLoRA as the default 4-bit recipe used to fine-tune Llama 3.1 70B on a single 48GB GPU.
- Unsloth and Axolotl both expose NF4 plus double quantisation as one-flag presets for QLoRA runs on consumer and rented GPUs.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does NF4 use Normal-distributed quantile buckets instead of uniform spacing?
Trained weights are approximately N(0, σ²), so most mass sits near zero. Quantile buckets place more levels where the density is high, minimising expected quantisation error under that prior.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Thinking the LoRA adapters are also 4-bit, or that a 4-bit base removes the need for gradient checkpointing. Both confuse weight memory with where learning and activations actually live.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.