Misconception: 'LoRA is just a quantization technique'

MCQ·Easy·4.0 · 0·~1 min·Relevant atCohere Databricks Meta Microsoft

Attempt it

Quick reference

TL;DR

LoRA is a parameter-efficient training method, not quantization. It trains tiny low-rank adapters over frozen weights. QLoRA adds the 4-bit part.

Memory aid

Easy to grasp

Imagine a huge cookbook you cannot rewrite. The first trick clips a few sticky notes onto the pages, and you only ever write on the notes. The cookbook stays exactly as it was, full size and full detail, while you jot your new recipe on the notes. The second trick is totally different: it reprints the whole cookbook in smaller, blurrier font to save shelf space. People mix these up because one popular method does both at once: it reprints the book small AND adds the sticky notes. But adding notes by itself is just the first trick. It never touches the font size of the original, and shrinking the font has nothing to do with the notes.

Key concepts

LoRA adapters Weight quantization QLoRA Parameter-efficient fine-tuning

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

3 min: define LoRA as parameter-efficient training, define quantization as precision reduction, separate the two axes, then explain how QLoRA combines a frozen 4-bit base with higher precision trainable adapters.

Side-by-side

Aspect	LoRA	Quantization
What it reduces	Number of trainable parameters	Bits used to store each weight
Base weights	Frozen, original precision	Stored in lower precision
Primary goal	Cheaper, faster fine-tuning	Smaller memory footprint
Touches precision	No	Yes
Example methods	LoRA, DoRA	GPTQ, AWQ, NF4

Real-world examples

Real products, models, and research that use this idea.

Hugging Face PEFT implements LoRA as adapter modules over a frozen base, with QLoRA enabled by passing a bitsandbytes 4-bit NF4 config separately.
QLoRA fine-tuning of Llama 4 variants on a single consumer GPU loads the base in 4-bit NF4 and trains bf16 adapters on top.
Unsloth and Axolotl both expose LoRA rank and quantization bit-width as distinct, independently set knobs in their training configs.
GPTQ and AWQ are inference-time quantization methods with no training step, illustrating that quantization is fully separable from LoRA.
OpenAI and Anthropic fine-tuning APIs apply parameter-efficient adapters server-side without exposing quantization, reinforcing that the two concepts are decoupled.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QIn QLoRA, how do gradients flow if the base weights are stored in 4-bit and frozen?

QWhy is rank the key LoRA knob while bit-width is the key quantization knob?

QHow does DoRA differ from LoRA, and does it involve quantization?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Conflating LoRA with QLoRA. LoRA cuts the number of trainable parameters; quantization cuts the precision of stored weights. They are orthogonal techniques that QLoRA happens to combine.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.