Match each weight quantization regime to what it buys you and where it breaks
Drag each answer to line up with its matching prompt
FP8 (E4M3 / E5M2) weights
Where INT4 typically breaks; mitigated by SmoothQuant / AWQ scaling
INT8 weights (W8)
~FP16 quality, 2× memory + bandwidth win, native on Hopper/Blackwell: minimal effort acceleration
INT4 weights (W4) via AWQ / GPTQ
Why smaller weights cut decode time directly (bandwidth bound regime)
Activation outliers at low bit
4× memory win, ~1-3 pt quality drop, needs group wise scales: biggest decode latency improvement
Decode arithmetic intensity ~1
Broadly safe with PTQ + per channel calibration; ~free on most architectures
FP8 is near-lossless and native on new hardware, INT8 is a mature safe PTQ default, and INT4 buys 4x compression but needs AWQ or GPTQ to survive activation outliers.
Imagine shipping a thick textbook by mail and paying by weight. FP8 prints it on slightly thinner paper, half the weight, and you can barely tell. INT8 uses cheaper but still readable paper, a well-tested trick that almost never goes wrong. INT4 shrinks the text to a quarter of the weight, which is great for postage, but now some fine print blurs and you need a careful reprinting process to keep it legible. The blurry parts are usually a few unusually important words that hate being squished. Special methods rescale those words first so they survive. Why care about weight at all? Because the mail cost here is reading numbers from memory, and lighter packages arrive faster.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Weight quantization is the practice of storing a model's parameters in fewer bits than the 16-bit float used during training. The payoff is twofold. The model occupies less memory, and during inference there are fewer bytes to stream from high-bandwidth memory per token. Because the decode phase of LLM serving is bound by memory bandwidth rather than raw compute, fewer bytes per weight read translates almost directly into faster token generation.
The interview question that this card encodes is a matching exercise, and matching exercises reward a clean mental model. The trap is to treat FP8, INT8, and INT4 as three settings of one dial. They are not. Each sits at a distinct point on a memory versus quality curve, each has a different relationship to the hardware, and only the most aggressive of them routinely needs special calibration to stay accurate.
It helps to anchor the whole picture in one number, the bytes per weight. FP16 spends two. FP8 and INT8 spend one. INT4 spends roughly a half plus a little overhead for scales. That single number sets both the memory footprint and, on decode, the latency. Everything else in this dive is about how far you can push the byte count down before quality gives way, and what tooling buys you another step.
This deep dive builds that model. It places the three precisions on the curve, explains what activation outliers are and why they break low-bit quantization, names the methods that fix them, and finally grounds everything in why decode being bandwidth-bound is the reason weight quantization is the single highest-leverage latency knob for token generation.
FP8: near-lossless and hardware-native
FP8 is an 8-bit floating-point format with two common variants. E4M3 uses four exponent bits and three mantissa bits, favoring precision, and is the usual choice for weights. E5M2 uses five exponent bits and two mantissa bits, favoring dynamic range, and shows up where wide ranges matter such as gradients.
The defining property of FP8 is that it keeps quality close to FP16 with very little effort. Because it is a float, it preserves a sensible distribution of values around zero rather than the uniform spacing of an integer grid. That makes it forgiving for the heavy-tailed magnitude distributions found in transformer weights.
The second defining property is hardware. NVIDIA Hopper and Blackwell tensor cores execute FP8 matrix multiplies natively at full rate. On those chips FP8 is close to a free 2x win in both memory footprint and bandwidth, with no integer dequantization step in the critical path. That is why it is the minimal-effort acceleration path when you control the silicon, and why it loses much of its appeal on older hardware without native FP8 units.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Precision | Memory win | Quality | Calibration effort | Hardware note |
|---|---|---|---|---|
| FP8 (E4M3/E5M2) | ~2x | Near FP16 | Minimal | Native Hopper / Blackwell |
| INT8 (W8) | ~2x | Near lossless | PTQ + per-channel | Broadly portable |
| INT4 (W4) AWQ/GPTQ | ~4x | 1-3 pt drop | Group-wise + outlier-aware | Best decode latency |
Real products, models, and research that use this idea.
- NVIDIA TensorRT-LLM serves FP8 weights and activations natively on H100 and B200, the default minimal-effort path on Hopper and Blackwell.
- vLLM and SGLang ship AWQ and GPTQ INT4 weight kernels so 70B-class models like Llama 4 fit on a single 80 GB GPU.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does INT4 need group-wise scales while INT8 can often use per-channel scales?
Think about how much dynamic range each scale must cover. At 4 bits there are only 16 levels, so one scale per channel cannot track local magnitude variation. Smaller groups, size 128, give each block its own scale and recover accuracy.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating all three precisions as interchangeable knobs. FP8, INT8, and INT4 sit at different points on the memory versus quality curve, and only INT4 routinely needs outlier-aware calibration.
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
- Lin et al., AWQ: Activation-aware Weight Quantization for LLM Compression
- Frantar et al., GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Xiao et al., SmoothQuant: Accurate and Efficient Post-Training Quantization for LLMs
- Micikevicius et al., FP8 Formats for Deep Learning
Same topic, related formats. Practice these next.