**FP4** stands for floating-point 4-bit. It is a standard number format with 1 sign bit, 2 exponent bits, and 1 mantissa bit, written E2M1. Only 16 distinct values exist per number, and those values are spread out using the same exponent-mantissa math that FP16 and FP32 use, just much coarser. NVIDIA Blackwell GPUs (B100, B200) include native FP4 tensor cores that multiply at FP4 directly, roughly doubling throughput versus FP8.
**NF4** stands for NormalFloat 4-bit. It also uses 4 bits per value, also has 16 levels, but those levels are not on a floating-point grid. They are placed so that for a unit-normal distribution of weights, each level gets about the same probability mass. NF4 was introduced in the QLoRA paper to compress frozen pretrained weights as tightly as possible while keeping useful precision near zero, where most LLM weights actually sit.
The niches differ. FP4 is for inference: hardware-accelerated low-precision matmul, mainly on Blackwell. NF4 is for QLoRA fine-tuning: storage-only compression of frozen weights, with the actual math happening in BF16 after dequantising.
**FP4** is a 4-bit floating-point format, typically E2M1 (sign + 2 exponent + 1 mantissa). Its 16 representable values are placed on a floating-point grid: clustered near zero and spaced exponentially outward. The key feature is that NVIDIA Blackwell (B100, B200, GB200) ships native FP4 tensor cores that multiply FP4 operands at full hardware speed, roughly doubling matmul throughput versus FP8.
**NF4** is a 4-bit non-uniform format introduced in QLoRA. Its 16 levels are chosen so that for a standard normal weight distribution, each bucket holds approximately equal probability mass. Pretrained LLM weights are roughly normally distributed, so NF4 wastes fewer levels on the tails and packs more resolution near zero where weights actually live.
The deployment niches barely overlap. FP4 is an inference-time matmul format on Blackwell, used to halve memory and double tensor-core throughput. NF4 is a storage-only format for frozen base weights in QLoRA fine-tuning: weights live in NF4 in HBM, get dequantised to BF16 on the fly during the forward pass, and LoRA adapters train in BF16 on top. NF4 was not designed for hardware-accelerated math; the dequant happens before the matmul.
Both exist because LLMs tolerate aggressive weight quantisation surprisingly well. They just answer different questions: FP4 asks how to make inference matmuls cheaper on new silicon, NF4 asks how to fit a frozen 70B model in 35 GB so a single 80 GB GPU can fine-tune it.
Quick reference
TL;DR
FP4 is a standard 4-bit floating-point format with native Blackwell tensor cores for fast inference matmul; NF4 is a non-uniform 4-bit format from QLoRA whose levels match a normal distribution for weight storage
Imagine compressing a list of numbers down to just 16 possible values each. You have two ways to pick those 16 levels. The first is to space them out using a clean math rule, the same way most hardware understands numbers. That picks a grid the chip can multiply with at full speed. The second is to look at the actual numbers you have, notice they bunch up near zero, and place more of your 16 levels in that crowded region so the compression hurts less. The first style is FP4 and runs fast on the newest chips. The second is NF4 and was invented to make fine-tuning huge models possible on a single GPU.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Inference deployment on Hopper-generation GPUs (H100, H200) Those chips do not have native FP4 tensor cores, so FP4 brings no throughput gain; FP8 is the right format on Hopper.
Full-precision fine-tuning that updates all weights QLoRA freezes the base weights and only trains LoRA adapters, so NF4 storage only fits workflows that keep the base frozen.
Latency-critical serving where any accuracy hit is unacceptable FP4 accuracy degradation, even after careful scaling, is larger than FP8 and may push tasks like coding or math past usable quality.
Small models where memory is not the constraint Both formats trade accuracy for memory; on a 7B model where BF16 fits with room to spare, neither buys you much.
Quick scan · 2–4 min
Everything important, quickly.
5 min: define FP4 as E2M1 with native Blackwell tensor cores, contrast NF4 as a non-uniform storage format from QLoRA whose levels match a normal distribution, walk through QLoRA's dequant-to-BF16 path, then compare deployment niches.
Real-world examples
Real products, models, and research that use this idea.
1NVIDIA Blackwell B200 ships native FP4 tensor cores, used by serving stacks like vLLM and TensorRT-LLM to run frontier models at higher throughput.
2QLoRA introduced NF4 in 2023 and remains the standard for memory-efficient fine-tuning in Hugging Face PEFT.
3Unsloth uses NF4 plus double quantisation to fit 70B fine-tunes on a single 80 GB H100 GPU.
4DeepSeek V4 serving stacks experimented with FP4 weights on B200 to roughly double decode throughput versus FP8.
5bitsandbytes library implements NF4 as the bnb_4bit_quant_type option that pairs with PEFT for QLoRA workflows.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does NF4 cluster more levels near zero than uniform 4-bit would?
Sign in or go Pro to see the approach for this follow-up.
QWhat does double quantisation add to NF4?
Sign in or go Pro to see the approach for this follow-up.
QWhen does FP4 inference actually beat FP8 in production today?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating FP4 and NF4 as interchangeable because both use 4 bits. They differ in where the 16 levels are placed and in whether dedicated tensor-core hardware multiplies them, which puts them in very different deployment niches.