Match each post-training quantization method to what it protects against
Drag each answer to line up with its matching prompt
AWQ
No calibration in the simple case: relies on FP8's wider dynamic range vs INT8
GPTQ
You need INT4 weight only quant for memory bound decode on a 70B class model
SmoothQuant
Migrates activation outliers into the weight side so W8A8 INT8 quant stays accurate
FP8 PTQ (cast and go)
Hessian aware rounding that minimizes second order reconstruction error layer by layer
Pick AWQ or GPTQ when…
Protects salient weight channels (found from activation magnitudes) via per channel scaling before INT4 rounding
Each quantization method fixes a different failure mode: GPTQ minimizes second-order error, AWQ protects salient weight channels, SmoothQuant migrates activation outliers, and FP8 leans on wider dynamic range.
Imagine compressing a detailed photo into far fewer colors. If you pick colors blindly, the important bright spots turn into ugly blotches. Different methods fix this differently. One studies which pixels matter most and rounds those carefully (GPTQ). One notices a few super-bright pixels dominate and scales them up before compressing so they survive (AWQ). One realizes the brightness problem actually lives in the lighting, not the photo, and shifts the glare onto the photo where it is easier to handle (SmoothQuant). And one just uses a color format that already spans a wider brightness range, so no special trick is needed (FP8). They share one goal, lower bits, but each guards against a different way the picture would otherwise be ruined.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Post-training quantization is the production method that lets a 70B model fit and serve on hardware that could never hold it in bf16. The decode phase of LLM inference is memory-bandwidth-bound: every generated token streams the weights and the KV cache out of HBM. Halving the bytes per weight roughly doubles the achievable decode throughput, which is why low-bit quantization is a standard part of serving stacks.
The difficulty is that you cannot just round. Transformer weights and activations have heavy-tailed distributions with rare but very large outliers. Uniform rounding to four or eight bits destroys exactly the values that carry the most signal. Each of these methods was designed to counter a specific way that naive rounding fails. Getting the pairs right means answering two independent questions for each method: what tensors does it quantize, and which distortion does it fight.
Why low-bit quantization is hard: outliers and salient channels
The naive picture of quantization is a uniform grid: take a tensor, find its min and max, and map the range onto a fixed set of integer levels. With four bits you have sixteen levels. With eight bits you have 256. This works when values are roughly uniform. Transformer tensors are not. A handful of weight channels and activation dimensions carry magnitudes orders of magnitude larger than the rest.
Those outliers force a hard tradeoff. If you set the quantization range wide enough to represent them, the step size becomes so coarse that ordinary values collapse to a few levels and lose all detail. If you clip the outliers to keep a fine step size, you discard the very dimensions that dominate the output. Either way, plain rounding badly hurts accuracy. The error is not random noise. It is concentrated in exactly the values that matter most.
There are two distinct outlier problems, and the matching depends on telling them apart. On the weight side, certain channels are salient: rounding them coarsely hurts the output disproportionately. On the activation side, certain feature dimensions grow very large at runtime. They are data-dependent, so a fix built into the weights needs care. Different methods address different sides, and that is why they pair with different descriptions.
There is also a bit-width axis on top. INT8 is forgiving enough that weights alone rarely need special methods, so the INT8 problem is mostly about activations. INT4 cuts the number of levels from 256 to 16, so even weights need protection. FP8 differs by trading some mantissa precision for exponent range.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Method | What it quantizes | Failure mode it fights | Typical bit width |
|---|---|---|---|
| GPTQ | Weights only | Cumulative layer reconstruction error | INT4 / INT3 |
| AWQ | Weights only | Loss of salient weight channels | INT4 |
| SmoothQuant | Weights and activations | Activation outliers blocking INT8 | INT8 (W8A8) |
| FP8 cast-and-go | Weights (and often activations) | Narrow dynamic range of INT8 | FP8 (E4M3) |
Real products, models, and research that use this idea.
- vLLM and SGLang ship AWQ and GPTQ INT4 kernels as a default way to serve Llama 3.1 70B on a single 80 GB GPU.
- The SmoothQuant paper reports W8A8 INT8 inference with up to 1.56x speedup and 2x memory reduction, and serves a 530B model on a single node.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does AWQ scale activations down when it scales salient weight channels up?
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating all four as interchangeable INT4 tricks. They differ on what they quantize (weights only vs weights and activations) and which failure mode they target (salient channels, layer error, or activation outliers).
60 second bullets to scan on the way to the call.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.