How does QLoRA fit a 65B fine-tune onto a single 48GB GPU?

Let's walk the memory math for a 65B QLoRA fine-tune on a 48 GB GPU. A 65B model in bf16 takes 130 GB just for the weights, which is nearly 3× our GPU. Full fine-tuning would add optimizer state and gradients on top, pushing past 600 GB and out of reach on one card. QLoRA layers four tricks to close that gap. First, NF4 is a 4-bit NormalFloat format tuned for the bell-curve shape of LLM weights, and it compresses the frozen base from 130 GB to about 33 GB. Second, double quantization shrinks the scaling factors that NF4 itself uses, shaving off another ~3 GB. Third, LoRA adds tiny trainable adapter layers in bf16, only ~200 million parameters, and the optimizer state lives only on those adapters, never on the frozen base. Fourth, paged optimizers let that optimizer state spill to CPU memory when GPU usage spikes. Add it up: ~30 GB base + ~0.4 GB adapter + ~1.6 GB optimizer + ~12 GB activations fits in 48 GB.

65B in bf16 weights alone is 130 GB, and full fine-tune would add another 520 GB of Adam state on top. QLoRA gets that down to roughly 45 GB on a single card via four tricks layered on top of LoRA's structural saving. Starting point: 65B in bf16 is 130 GB of weights. Full fine-tune adds Adam state at 8 bytes per parameter. That's another 520 GB on top, so we're nowhere near a single 48 GB card. The first trick is NF4. NormalFloat-4 quantizes the frozen base from 2 bytes per parameter to roughly 0.5 bytes, dropping the base from 130 GB to about 33 GB. It uses quantile buckets matched to the Gaussian shape of LLM weights, so quality loss is small. The second trick is double quantization. The per-block FP32 scales used by NF4 themselves get quantized, saving roughly 0.4 bits per parameter. Base lands around 30 GB. The third trick, and the real win, is LoRA on top. Adapters at rank 64 are about 200M parameters in bf16, around 0.4 GB. Adam state lives only on the adapters, so optimizer memory drops from 520 GB to about 1.6 GB. That's the structural saving. The fourth trick is paged optimizers. When activations peak during backward, optimizer state pages out to CPU over CUDA unified memory, then pages back. Add checkpointed activations (~12 GB) and headroom, totals around 45-50 GB. The trade-off is dequant overhead at every matmul, but for training that's acceptable.