Explain what gets quantized the second time in QLoRA's double-quantization step

**QLoRA** stores the frozen base model's weights in a 4-bit format called NF4. The headline number is 4 bits per weight, but the actual cost is slightly higher. NF4 uses block-wise quantization. It splits the weights into blocks of 64, computes a scale factor for each block in fp32, and stores both the 4-bit codes and the scale together. The scale is what lets the dequantizer reconstruct the actual weight value from the 4-bit code. One fp32 scale per 64 weights works out to 32 bits divided by 64, which is 0.5 bits per weight of overhead on top of the 4-bit codes. So the real cost is 4.5 bits per weight, not 4. **Double quantization** is the trick that compresses those scale factors themselves. The fp32 scales are clustered and quantized to 8 bits, with a second-level scale per group of scales kept in fp32. The math works out to roughly 0.13 bits per weight of overhead instead of 0.5, a saving of about 0.4 bits per weight. On a 7-billion-parameter base model that is about 350 MB of recovered memory. Not huge, but free and applied every time QLoRA loads the base, so the feature is almost always on by default in modern recipes.

**Double quantization** is the QLoRA optimization that compresses the per-block scale factors that NF4 itself leaves uncompressed. The weights are not quantized a second time; only their metadata is. The setup: NF4 uses block-wise quantization with a block size of 64. Each block of 64 weights gets one fp32 scale factor stored alongside it, used at dequant time to map the 4-bit codes back to floating-point values. One fp32 scale per 64 weights is 0.5 bits per weight of metadata overhead on top of the 4-bit codes. The optimization: those fp32 scales are themselves quantized in a second pass. Scales are grouped into blocks of 256, each block gets a second-level fp32 scale, and the individual scales within a block are quantized to 8-bit fp values. The new overhead is 8 bits per scale divided by 64 weights per block, plus a 32-bit second-level scale per 256 first-level scales, working out to roughly 0.13 bits per weight. The net saving is about 0.4 bits per weight on the frozen base. For a 70B model that translates to roughly 3.5 GB of GPU memory recovered, with negligible impact on numerical quality because the scales tolerate quantization much better than the weights do. Modern QLoRA recipes enable it by default with `bnb_4bit_use_double_quant=True`, and there is essentially no reason to turn it off.