Top 100 · Inference Optimization48 of 100

Match each 4-bit format to its niche in 2026 inference and fine-tuning.

Match pairs·Easy·4.0 · 0·~2 min

Attempt it

FP4 (E2M1)

NF4 (NormalFloat 4)

Typical FP4 deployment

Typical NF4 deployment

**FP4** stands for floating-point 4-bit. It is a standard number format with 1 sign bit, 2 exponent bits, and 1 mantissa bit, written E2M1. Only 16 distinct values exist per number, and those values are spread out using the same exponent-mantissa math that FP16 and FP32 use, just much coarser. NVIDIA Blackwell GPUs (B100, B200) include native FP4 tensor cores that multiply at FP4 directly, roughly doubling throughput versus FP8. **NF4** stands for NormalFloat 4-bit. It also uses 4 bits per value, also has 16 levels, but those levels are not on a floating-point grid. They are placed so that for a unit-normal distribution of weights, each level gets about the same probability mass. NF4 was introduced in the QLoRA paper to compress frozen pretrained weights as tightly as possible while keeping useful precision near zero, where most LLM weights actually sit. The niches differ. FP4 is for inference: hardware-accelerated low-precision matmul, mainly on Blackwell. NF4 is for QLoRA fine-tuning: storage-only compression of frozen weights, with the actual math happening in BF16 after dequantising.

**FP4** is a 4-bit floating-point format, typically E2M1 (sign + 2 exponent + 1 mantissa). Its 16 representable values are placed on a floating-point grid: clustered near zero and spaced exponentially outward. The key feature is that NVIDIA Blackwell (B100, B200, GB200) ships native FP4 tensor cores that multiply FP4 operands at full hardware speed, roughly doubling matmul throughput versus FP8. **NF4** is a 4-bit non-uniform format introduced in QLoRA. Its 16 levels are chosen so that for a standard normal weight distribution, each bucket holds approximately equal probability mass. Pretrained LLM weights are roughly normally distributed, so NF4 wastes fewer levels on the tails and packs more resolution near zero where weights actually live. The deployment niches barely overlap. FP4 is an inference-time matmul format on Blackwell, used to halve memory and double tensor-core throughput. NF4 is a storage-only format for frozen base weights in QLoRA fine-tuning: weights live in NF4 in HBM, get dequantised to BF16 on the fly during the forward pass, and LoRA adapters train in BF16 on top. NF4 was not designed for hardware-accelerated math; the dequant happens before the matmul. Both exist because LLMs tolerate aggressive weight quantisation surprisingly well. They just answer different questions: FP4 asks how to make inference matmuls cheaper on new silicon, NF4 asks how to fit a frozen 70B model in 35 GB so a single 80 GB GPU can fine-tune it.

Quick reference

TL;DR

FP4 is a standard 4-bit floating-point format with native Blackwell tensor cores for fast inference matmul; NF4 is a non-uniform 4-bit format from QLoRA whose levels match a normal distribution for weight storage

Memory aid

Easy to grasp

Imagine compressing a list of numbers down to just 16 possible values each. You have two ways to pick those 16 levels. The first is to space them out using a clean math rule, the same way most hardware understands numbers. That picks a grid the chip can multiply with at full speed. The second is to look at the actual numbers you have, notice they bunch up near zero, and place more of your 16 levels in that crowded region so the compression hurts less. The first style is FP4 and runs fast on the newest chips. The second is NF4 and was invented to make fine-tuning huge models possible on a single GPU.

Key concepts

Quantization FP4 (E2M1)NF4 (NormalFloat 4)QLoRA

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 min: define FP4 as E2M1 with native Blackwell tensor cores, contrast NF4 as a non-uniform storage format from QLoRA whose levels match a normal distribution, walk through QLoRA's dequant-to-BF16 path, then compare deployment niches.

Real-world examples

Real products, models, and research that use this idea.

NVIDIA Blackwell B200 ships native FP4 tensor cores, used by serving stacks like vLLM and TensorRT-LLM to run frontier models at higher throughput.
QLoRA introduced NF4 in 2023 and remains the standard for memory-efficient fine-tuning in Hugging Face PEFT.
Unsloth uses NF4 plus double quantisation to fit 70B fine-tunes on a single 80 GB H100 GPU.
DeepSeek V4 serving stacks experimented with FP4 weights on B200 to roughly double decode throughput versus FP8.
bitsandbytes library implements NF4 as the bnb_4bit_quant_type option that pairs with PEFT for QLoRA workflows.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does NF4 cluster more levels near zero than uniform 4-bit would?

QWhat does double quantisation add to NF4?

QWhen does FP4 inference actually beat FP8 in production today?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating FP4 and NF4 as interchangeable because both use 4 bits. They differ in where the 16 levels are placed and in whether dedicated tensor-core hardware multiplies them, which puts them in very different deployment niches.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.