Match each 4-bit format to its niche in inference and fine-tuning.

Match pairs·Easy·4.0 · 0·~2 min·Asked atContextual Ai Uipath·Relevant atNVIDIA

Attempt it

Drag each answer to line up with its matching prompt

FP4 (E2M1)

4-bit format whose 16 quantization levels are spaced to match a normal distribution, designed for weight storage in QLoRA

NF4 (NormalFloat 4)

QLoRA fine-tuning where the frozen base weights live in NF4 and LoRA adapters train in BF16

Typical FP4 deployment

Standard 4-bit floating point format with native Blackwell tensor core support, used for low bit inference matmul

Typical NF4 deployment

Hardware accelerated low precision serving on B100/B200 GPUs

**FP4** stands for floating-point 4-bit. It is a standard number format with 1 sign bit, 2 exponent bits, and 1 mantissa bit, written E2M1. Only 16 distinct values exist per number, and those values are spread out using the same exponent-mantissa math that FP16 and FP32 use, just much coarser. NVIDIA Blackwell GPUs (B100, B200) include native FP4 tensor cores that multiply at FP4 directly, roughly doubling throughput versus FP8. **NF4** stands for NormalFloat 4-bit. It also uses 4 bits per value, also has 16 levels, but those levels are not on a floating-point grid. They are placed so that for a unit-normal distribution of weights, each level gets about the same probability mass. NF4 was introduced in the QLoRA paper to compress frozen pretrained weights as tightly as possible while keeping useful precision near zero, where most LLM weights actually sit. The niches differ. FP4 is for inference: hardware-accelerated low-precision matmul, mainly on Blackwell. NF4 is for QLoRA fine-tuning: storage-only compression of frozen weights, with the actual math happening in BF16 after dequantising.

**FP4** is a 4-bit floating-point format, typically E2M1 (sign + 2 exponent + 1 mantissa). Its 16 representable values are placed on a floating-point grid: clustered near zero and spaced exponentially outward. The key feature is that NVIDIA Blackwell (B100, B200, GB200) ships native FP4 tensor cores that multiply FP4 operands at full hardware speed, roughly doubling matmul throughput versus FP8. **NF4** is a 4-bit non-uniform format introduced in QLoRA. Its 16 levels are chosen so that for a standard normal weight distribution, each bucket holds approximately equal probability mass. Pretrained LLM weights are roughly normally distributed, so NF4 wastes fewer levels on the tails and packs more resolution near zero where weights actually live. The deployment niches barely overlap. FP4 is an inference-time matmul format on Blackwell, used to halve memory and double tensor-core throughput. NF4 is a storage-only format for frozen base weights in QLoRA fine-tuning: weights live in NF4 in HBM, get dequantised to BF16 on the fly during the forward pass, and LoRA adapters train in BF16 on top. NF4 was not designed for hardware-accelerated math; the dequant happens before the matmul. Both exist because LLMs tolerate aggressive weight quantisation surprisingly well. They solve different problems. FP4 makes inference matmuls cheaper on new hardware. NF4 lets a frozen 70B model fit in about 35 GB so a single 80 GB GPU can fine-tune it.

TL;DR

FP4 is a standard 4-bit floating-point format with native Blackwell tensor cores for fast inference matmul; NF4 is a non-uniform 4-bit format from QLoRA whose levels match a normal distribution for weight storage.

Memory aid

Easy to grasp

Imagine compressing a list of numbers down to just 16 possible values each. You have two ways to pick those 16 levels. The first is to space them out using a clean math rule, the same way most hardware understands numbers. That picks a grid the chip can multiply with at full speed. The second is to look at the actual numbers you have, notice they bunch up near zero, and place more of your 16 levels in that crowded region so the compression hurts less. The first style is FP4 and runs fast on the newest chips. The second is NF4 and was invented to make fine-tuning huge models possible on a single GPU.

Key concepts

Quantization FP4 (E2M1)NF4 (NormalFloat 4)QLoRA

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

4-bit quantisation is one of the most consequential techniques in LLM systems. Going from BF16 down to 4 bits cuts weight memory and weight traffic by about 4x, and on Blackwell doubles peak matmul throughput over FP8. But not all 4-bit formats are the same, and the two most often confused (FP4 and NF4) solve different problems and live in different niches.

FP4 is a hardware-friendly floating-point format for inference matmul on Blackwell-generation GPUs. NF4 is a statistically-tuned storage format for QLoRA fine-tuning of frozen base models. They share a bit width and almost nothing else. Confusing them is one of the most common mistakes in low-bit quantisation, and the fix is a clear mental model of where the 16 representable levels come from in each case.

FP4: the floating-point format with hardware support

FP4 in the E2M1 layout has 1 sign bit, 2 exponent bits, and 1 mantissa bit. With 4 total bits there are at most 16 representable values, and they sit on a floating-point grid: ±0, ±0.5, ±1, ±1.5, ±2, ±3, ±4, ±6. The spacing grows with magnitude, the floating-point shape.

This grid is the natural format for tensor cores. Hardware multipliers take two operands and produce a product in a fixed circuit. The same physical multiplier that handles FP16 can be reused for FP4 with much wider parallelism, since each operand is 4 bits instead of 16. NVIDIA Blackwell added native FP4 tensor cores in 2024, providing roughly 2x the FP8 throughput on the same silicon, or 4x BF16. Because the E2M1 range is so small, Blackwell runs FP4 with block scales: NVFP4 uses one FP8 E4M3 scale per 16 values plus an FP32 per-tensor scale, and MXFP4 uses one power-of-two E8M0 scale per 32 values.

For LLM inference, FP4 weight quantisation halves memory versus FP8. On Blackwell that memory saving combines with the throughput gain to make FP4 the most aggressive viable matmul format for serving frontier models. Accuracy degradation is real, however; FP4 quantisation needs those fine-grained scales, careful activation handling, and sometimes mixed precision for sensitive layers.

NF4: the statistical format for QLoRA

Why NF4 cannot be multiplied directly by hardware

Niche comparison: inference versus fine-tuning

Accuracy trade-offs and where each starts to break

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

NVIDIA Blackwell B200 ships native FP4 tensor cores, used by serving stacks like vLLM and TensorRT-LLM to run frontier models at higher throughput.
QLoRA introduced NF4 in 2023, and it remains the standard format for memory-efficient fine-tuning in Hugging Face PEFT.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does NF4 cluster more levels near zero than uniform 4-bit would?

Pretrained LLM weights are approximately normally distributed around zero. Uniform 4-bit spacing wastes resolution on the rare large-magnitude tails. NF4 derives its levels from N(0,1) quantiles so each level catches roughly equal probability mass, putting more granularity where the weights actually live.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating FP4 and NF4 as interchangeable because both use 4 bits. They differ in where the 16 levels are placed and in whether dedicated tensor-core hardware multiplies them, which puts them in very different deployment niches.

60 second bullets to scan on the way to the call.

The bit layout of FP4 (E2M1: 1 sign, 2 exponent, 1 mantissa)
How NF4 derives its 16 levels from quantiles of a normal distribution

Primary sources. Browse if you want the original framing.