Which statements about FP8, INT8 and INT4 weight quantization are correct?

**Weight quantization** stores a model's parameters in fewer bits than the FP16 default. FP8 uses 8 bits, INT8 uses 8, and INT4 uses 4. Fewer bits means fewer bytes, so the model takes less memory and is faster to read off the GPU. Why does reading speed matter? During token generation, the decode phase, the GPU is mostly busy streaming weights from memory rather than doing math. This is called being memory bandwidth bound. Halving the bytes per weight roughly halves the streaming time, so latency improves nearly in proportion to the bit reduction. The catch is quality. FP8 keeps a floating-point format, so it tolerates the wide range of values in weights and drops almost no accuracy. INT4 is far more aggressive. Naive rounding to 4 bits damages the model, so methods like AWQ and GPTQ add per-group scale factors and protect the most important weight channels. Two tempting but false claims appear in the options. One says INT8 needs special retraining; it does not, since calibration is enough. The other says every GPU accelerates INT4 natively; older generations often dequantize back to FP16 to run the matmul.

**Weight quantization** trades numeric precision for fewer bytes per parameter. FP8 gives about 2x savings versus FP16. INT4 gives about 4x. The savings scale with the bit reduction. The latency win comes from bandwidth, not arithmetic. Decode is memory-bound, so the GPU spends most cycles streaming weights from HBM. Fewer bytes per weight means fewer bytes streamed per token. So decode latency drops nearly linearly with bit-width, until software overhead or dequantization cost dominates. Quality behaves differently by format. FP8 keeps an exponent, so it handles the dynamic range of weights with near-zero drop on Hopper and Blackwell tensor cores. INT8 post-training quantization with per-channel scales is safe and standard. It does not need quantization-aware training. Lower bits demand more care. INT4 needs calibration and outlier handling. AWQ protects salient channels. GPTQ uses Hessian-aware rounding. Naive round to nearest at 4 bits degrades sharply. Hardware support also varies. Native INT4 tensor-core paths are generation-specific. So many INT4 deployments run W4A16, dequantizing to FP16 for the matmul.