Match each GPU generation to the quantization format that lights up its native tensor cores.

Attempt it

H100 / H200 (Hopper)

B200 / GB200 (Blackwell)

INT4 weights via AWQ / GPTQ

INT8 weight + INT8 activation (W8A8)

FP16 / BF16 (no quantization)

**Each NVIDIA generation lowers the precision floor of its tensor cores.** Hopper added native FP8 (two flavors: E4M3 for weights and forward, E5M2 for gradients). Blackwell added native FP4 on top of FP8. The match for each option: **H100 / H200 (Hopper) maps to FP8.** This is the production sweet spot for the Hopper generation. FP8 doubles BF16 throughput on the same hardware. The toolchain (TensorRT-LLM, vLLM, Marlin) is mature. Every modern serving stack on H100 defaults to FP8. **B200 / GB200 (Blackwell) maps to FP4.** This is the new native floor on Blackwell. FP4 doubles FP8 throughput again. Running FP8 on Blackwell still works (FP8 is supported), but you leave half the chip's headline throughput on the table. **INT4 weights via AWQ / GPTQ** is portable. These weight-only schemes pack the weights at 4 bits but keep activations at 16 bits (W4A16). The advantage is hardware portability: works on Hopper, Blackwell, and even older Ampere with minor adjustments. Useful when your deployment target spans GPU generations. **W8A8 (INT8 weight + INT8 activation)** is the pre-Hopper baseline. A100-era quantization. On H100 it runs but loses to FP8 because FP8 has wider dynamic range with similar throughput. **FP16 / BF16** is the unquantized reference. Use it for accuracy floor measurements and ablations, but it leaves half the throughput on Hopper and three-quarters on Blackwell.

**The matches map each GPU generation to its native low-precision floor.** Each NVIDIA data-center generation introduces a new lowest-precision tensor-core format, doubling throughput for compute-bound work compared to the previous floor. **Hopper (H100 / H200) to FP8.** Hopper's fourth-generation tensor cores added native FP8 in two variants: E4M3 (4 exponent + 3 mantissa, for weights and forward activations) and E5M2 (5 exponent + 2 mantissa, for gradients). Peak FP8 throughput on H100 SXM is 1980 TFLOP/s versus 990 BF16. The Transformer Engine software stack made FP8 mixed precision plug and play in 2023, and FP8 is the production default on Hopper. **Blackwell (B200 / GB200) to FP4.** Blackwell's fifth-generation tensor cores add native FP4 alongside the existing FP8 path. B200 hits roughly 4500 TFLOP/s FP8 and 9000 TFLOP/s FP4. The FP4 path is what unlocks Blackwell's headline 2-3x throughput claims over Hopper. Running FP8 on Blackwell is supported and accurate, but leaves the FP4 throughput unused. **INT4 weights (AWQ / GPTQ) as portable W4A16.** These weight-only quantization schemes pack weights at 4 bits and run activations at 16 bits. Tensor cores process the activations in BF16 or FP16; the weights are dequantized just in time before the matmul. Throughput is bandwidth-bound (weight bytes are 4x smaller, so reads are 4x faster) but compute stays at BF16 rates. Portable across GPU generations because the math runs in BF16, regardless of whether FP8 or FP4 hardware exists. **W8A8 as the pre-Hopper baseline.** INT8 weight + INT8 activation was the A100-era quantization story. On H100 it works but does not exploit the new FP8 path. FP8 maintains wider dynamic range than INT8 (the 4-bit exponent gives more headroom for outlier activations) with similar throughput, so FP8 dominates W8A8 on Hopper. **FP16 / BF16 as the reference.** Useful for measuring quantization accuracy loss and as a baseline for ablations.

Quick reference

TL;DR

Hopper's native floor is FP8; Blackwell adds FP4 on top. INT4-weight (AWQ/GPTQ) is portable; W8A8 is pre-Hopper; BF16 is the reference baseline.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Each new GPU generation lowers the floor of what numbers it can multiply natively. The older generation could only count down to a certain precision before it had to fake it. The new generation introduces a smaller, faster unit. Picking the right floor for your hardware is like picking the right gear in a car. Too high a gear and the engine bogs down, too low and you redline. FP8 is the right gear for Hopper. FP4 is the right gear for Blackwell. Using FP8 on Blackwell is like driving in third when fourth would be smoother. Using FP4 on Hopper does not work at all because there is no native FP4 unit, so the GPU has to simulate it slowly.

Key concepts

Quantization FP8 precision FP4 precision Hopper architecture Blackwell architecture

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

5 min: walk the generational floor pattern from Hopper (FP8) to Blackwell (FP4), distinguish E4M3 from E5M2, explain W4A16 portability, contrast W8A8 against FP8 on H100, and close with the practical hardware-driven decision rule.

Real-world examples

Real products, models, and research that use this idea.

Llama 4 Maverick on H200 ships with native FP8 weights via the Transformer Engine for production serving.
DeepSeek V4 serving on H100 uses FP8 W8A8 with custom kernels for further throughput gains.
Anthropic Claude Opus 4.7 and OpenAI GPT-5.5 are widely understood to use FP8 in production on Hopper-class clusters.
NVIDIA's TensorRT-LLM v0.12+ ships a turnkey FP4 path for B200 inference, including MX-FP4 micro-scaling support.
Hugging Face's Marlin kernels enable W4A16 (INT4 weight, FP16 activation) on H100 for cost-sensitive deployments needing weight portability.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does the Transformer Engine pick per-tensor scale factors for FP8?

Sign in or go Pro to see the approach for this follow-up.

QWhy does Blackwell's FP4 require micro-scaling (MX-FP4) rather than per-tensor scaling like FP8?

Sign in or go Pro to see the approach for this follow-up.

QWhat is the practical accuracy gap between FP8, FP4, and W4A16 on a modern LLM?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Picking the same quantization for every GPU generation. FP8 on Blackwell wastes throughput; FP4 on Hopper has no native path; W8A8 on H100 underperforms FP8.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.