Match each 4-bit format to its niche in 2026 inference and fine-tuning.

Match pairs·Easy·4.0 · 0·~2 min·Asked atAccenture Contextual Ai Uipath·Relevant atNVIDIA

Attempt it

Drag each answer to line up with its matching prompt

FP4 (E2M1)

4-bit format whose 16 quantization levels are spaced to match a normal distribution, designed for weight storage in QLoRA

NF4 (NormalFloat 4)

QLoRA fine-tuning where the frozen base weights live in NF4 and LoRA adapters train in BF16

Typical FP4 deployment

Standard 4-bit floating point format with native Blackwell tensor core support, used for low bit inference matmul

Typical NF4 deployment

Hardware accelerated low precision serving on B100/B200 GPUs

**FP4** stands for floating-point 4-bit. It is a standard number format with 1 sign bit, 2 exponent bits, and 1 mantissa bit, written E2M1. Only 16 distinct values exist per number, and those values are spread out using the same exponent-mantissa math that FP16 and FP32 use, just much coarser. NVIDIA Blackwell GPUs (B100, B200) include native FP4 tensor cores that multiply at FP4 directly, roughly doubling throughput versus FP8. **NF4** stands for NormalFloat 4-bit. It also uses 4 bits per value, also has 16 levels, but those levels are not on a floating-point grid. They are placed so that for a unit-normal distribution of weights, each level gets about the same probability mass. NF4 was introduced in the QLoRA paper to compress frozen pretrained weights as tightly as possible while keeping useful precision near zero, where most LLM weights actually sit. The niches differ. FP4 is for inference: hardware-accelerated low-precision matmul, mainly on Blackwell. NF4 is for QLoRA fine-tuning: storage-only compression of frozen weights, with the actual math happening in BF16 after dequantising.

**FP4** is a 4-bit floating-point format, typically E2M1 (sign + 2 exponent + 1 mantissa). Its 16 representable values are placed on a floating-point grid: clustered near zero and spaced exponentially outward. The key feature is that NVIDIA Blackwell (B100, B200, GB200) ships native FP4 tensor cores that multiply FP4 operands at full hardware speed, roughly doubling matmul throughput versus FP8. **NF4** is a 4-bit non-uniform format introduced in QLoRA. Its 16 levels are chosen so that for a standard normal weight distribution, each bucket holds approximately equal probability mass. Pretrained LLM weights are roughly normally distributed, so NF4 wastes fewer levels on the tails and packs more resolution near zero where weights actually live. The deployment niches barely overlap. FP4 is an inference-time matmul format on Blackwell, used to halve memory and double tensor-core throughput. NF4 is a storage-only format for frozen base weights in QLoRA fine-tuning: weights live in NF4 in HBM, get dequantised to BF16 on the fly during the forward pass, and LoRA adapters train in BF16 on top. NF4 was not designed for hardware-accelerated math; the dequant happens before the matmul. Both exist because LLMs tolerate aggressive weight quantisation surprisingly well. They just answer different questions: FP4 asks how to make inference matmuls cheaper on new silicon, NF4 asks how to fit a frozen 70B model in 35 GB so a single 80 GB GPU can fine-tune it.

TL;DR

FP4 is a standard 4-bit floating-point format with native Blackwell tensor cores for fast inference matmul; NF4 is a non-uniform 4-bit format from QLoRA whose levels match a normal distribution for weight storage

Memory aid

Easy to grasp

Imagine compressing a list of numbers down to just 16 possible values each. You have two ways to pick those 16 levels. The first is to space them out using a clean math rule, the same way most hardware understands numbers. That picks a grid the chip can multiply with at full speed. The second is to look at the actual numbers you have, notice they bunch up near zero, and place more of your 16 levels in that crowded region so the compression hurts less. The first style is FP4 and runs fast on the newest chips. The second is NF4 and was invented to make fine-tuning huge models possible on a single GPU.

Key concepts

Quantization FP4 (E2M1)NF4 (NormalFloat 4)QLoRA

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

4-bit quantisation is one of the most consequential techniques in modern LLM systems. Going from BF16 down to 4 bits cuts memory by 4x, halves bandwidth pressure, and on the latest hardware doubles compute throughput. But not all 4-bit formats are the same, and the two that matter today (FP4 and NF4) solve different problems and live in different niches.

This deep dive separates the two cleanly. FP4 is a hardware-friendly floating-point format for inference matmul on Blackwell-generation GPUs. NF4 is a statistically-tuned storage format for QLoRA fine-tuning of frozen base models. They share a bit width and almost nothing else.

The goal by the end is to be able to tell, looking at any 4-bit LLM workflow, whether FP4 or NF4 is the right format, why, and what the trade-offs are. Confusing them is one of the most common mistakes in low-bit quantisation discussions, and the fix is a clear mental model of where the 16 representable levels come from in each case.

FP4: the floating-point format with hardware support

FP4 in the E2M1 layout has 1 sign bit, 2 exponent bits, and 1 mantissa bit. With 4 total bits there are at most 16 representable values, and they sit on a floating-point grid: ±0, ±0.5, ±1, ±1.5, ±2, ±3, ±4, ±6 (the exact set varies by spec, but the shape is exponential).

This grid is the natural format for tensor cores. Hardware multipliers consume two operands and produce a product in a fixed circuit. The same physical multiplier that handles FP16 can be reused for FP4 with much wider parallelism, since each operand is 4 bits instead of 16. NVIDIA Blackwell adds native FP4 tensor cores in 2024, providing roughly 2x the FP8 throughput on the same silicon, or 4x BF16.

For LLM inference, FP4 weight quantisation halves memory versus FP8. On Blackwell that memory saving combines with the throughput gain to make FP4 the most aggressive viable matmul format for serving frontier models. Accuracy degradation is real, however; FP4 quantisation usually needs per-channel scales, careful activation handling, and sometimes mixed precision for sensitive layers.

NF4: the statistical format for QLoRA

Why NF4 cannot be multiplied directly by hardware

Niche comparison: inference versus fine-tuning

Accuracy trade-offs and where each starts to break

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

NVIDIA Blackwell B200 ships native FP4 tensor cores, used by serving stacks like vLLM and TensorRT-LLM to run frontier models at higher throughput.
QLoRA introduced NF4 in 2023 and remains the standard for memory-efficient fine-tuning in Hugging Face PEFT.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does NF4 cluster more levels near zero than uniform 4-bit would?

Pretrained LLM weights are approximately normally distributed around zero. Uniform 4-bit spacing wastes resolution on the rare large-magnitude tails. NF4 derives its levels from N(0,1) quantiles so each level catches roughly equal probability mass, putting more granularity where the weights actually live.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Treating FP4 and NF4 as interchangeable because both use 4 bits. They differ in where the 16 levels are placed and in whether dedicated tensor-core hardware multiplies them, which puts them in very different deployment niches.

60 second bullets to scan on the way to the call.

The bit layout of FP4 (E2M1: 1 sign, 2 exponent, 1 mantissa)
How NF4 derives its 16 levels from quantiles of a normal distribution

Primary sources. Browse if you want the original framing.