Why does QLoRA use NF4 instead of uniform INT4?

Attempt it

Why does QLoRA quantise the frozen base model with NF4 rather than uniform INT4? What assumption does NF4 exploit, and why does that assumption hold for LLM weights?

Free · 2 AI evals / day

Quick reference

TL;DR

Trained transformer weights are roughly zero-mean Gaussian, so NF4 places its 16 levels at Normal quantiles. That puts resolution where the mass is and beats uniform INT4 at the same 4 bits.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Picture sorting 16 buckets to catch raindrops, where almost all the rain falls in one small spot. Uniform INT4 spaces the buckets evenly, so most sit in dry corners and waste capacity. NF4 looks at the rain map first and crowds buckets where the drops actually land, near the centre, with just a few out in the dry tails. Trained model weights behave like that rain: tons of values near zero, very few far out. By matching its buckets to the bell-curve shape, NF4 catches the values more precisely than even spacing ever could, using the exact same number of buckets. Same storage budget, smaller average rounding error.

Key concepts

QLoRA NF4 quantization Gaussian weight distribution Block-wise quantization

Quick scan · 2–4 min

Everything important, quickly.

5 min: 16-level budget framing + why weights are Gaussian + Normal-quantile placement + the optimality argument + per-block scale and double quantisation + when NF4 stops being the right choice.

Side-by-side

Aspect	Uniform INT4	NF4 (NormalFloat-4)
Level placement	Equal-width grid across the range	Quantiles of a unit Normal distribution
Optimal for	Uniformly distributed values	Zero-mean Gaussian values
Resolution near zero	Same as everywhere else	Highest, where most weights live
Tail handling	Wastes many levels on empty edges	Few levels, matching sparse tails
Error on real LLM weights	Higher at the same 4 bits	Lower at the same 4 bits

Real-world examples

Real products, models, and research that use this idea.

QLoRA's original release fine-tuned 65B models on a single 48GB GPU using NF4, and the technique now ships in Hugging Face bitsandbytes for fine-tuning Llama 4 and DeepSeek V4 bases.
Hugging Face PEFT plus bitsandbytes loads a 4-bit NF4 base and trains LoRA adapters on top, the default recipe in most 2026 open-weight fine-tuning tutorials.
Unsloth's fast fine-tuning stack uses NF4 double quantisation to fit 70B-class models on consumer 24GB cards while keeping quality near the bf16 baseline.
Axolotl exposes a 4-bit NF4 load flag as a one-line config option, used across community fine-tunes of Mistral, Qwen, and Llama 4 checkpoints.
Ollama and many local-inference stacks store K-quant variants whose design follows the same distribution-matching logic NF4 popularised for 4-bit weights.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does placing levels at Normal quantiles minimise expected quantisation error for a Gaussian source?

Sign in or go Pro to see the approach for this follow-up.

QWhat goes wrong if a single global scale is used instead of QLoRA's per-block scale of 64 weights?

Sign in or go Pro to see the approach for this follow-up.

QHow does double quantisation recover overhead, and what does it cost?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.