Which scenario most justifies the extra cost of QAT over PTQ for inference quantization?

Attempt it

Quick reference

TL;DR

QAT only earns its training cost in the sub-4-bit regime where PTQ accuracy collapses and the deployment cannot tolerate that loss; otherwise cheap PTQ wins.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine shrinking a detailed painting onto a tiny postage stamp. PTQ paints the full canvas first, then shrinks it and hopes the picture survives. For a mild shrink that works fine. But shrink it really hard and the picture turns to mush, because the painter never planned for so little space. QAT lets the painter practice on the tiny stamp the whole time, so they learn to place every stroke where it still reads clearly after shrinking. That practice costs a lot of extra effort. You only bother when the shrink is so severe that the careless approach produces garbage and you absolutely need the picture to stay sharp.

Key concepts

Quantization Quantization-aware training Post-training quantization

Quick scan · 2–4 min

Everything important, quickly.

3 min: PTQ versus QAT mechanism + cost gap + the bit-width cliff (INT8/INT4 fine, sub-4-bit collapses) + why QAT buys accuracy not speed + the on-device case.

Side-by-side

Dimension	PTQ	QAT
When applied	After training, round finished weights	During training, simulate rounding
Cost	Minutes, a few hundred calibration samples	Full fine-tune, gradients, GPU hours
Best bit width	INT8 and usually INT4 (AWQ, GPTQ)	2-bit and 3-bit where PTQ collapses
What it buys	Cheap, fast, default deployment path	Accuracy recovery at aggressive low bits
Inference speed	Same as QAT at equal bit width	Same as PTQ at equal bit width

Real-world examples

Real products, models, and research that use this idea.

Llama 4 ships official PTQ INT4 and FP8 checkpoints because the quality loss at 4-bit is negligible and no fine-tune is needed.
AWQ and GPTQ are the de facto PTQ methods in vLLM and TensorRT-LLM for serving INT4 weights in 2026.
On-device assistants on phones and laptops push to 2-bit or 3-bit and lean on QAT-style training to keep accuracy usable under a hard memory budget.
PyTorch and TensorRT both expose QAT fake-quant modules used when sub-4-bit deployments cannot tolerate PTQ's accuracy collapse.
DeepSeek V4 and Mistral Large 3 are distributed primarily as PTQ FP8 weights for cost-efficient datacenter serving.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does PTQ accuracy collapse below 4 bits rather than degrade smoothly?

Sign in or go Pro to see the approach for this follow-up.

QHow does QAT backpropagate through the non-differentiable rounding operation?

Sign in or go Pro to see the approach for this follow-up.

QIf both produce the same bit width, why might QAT weights still serve differently?

Sign in or go Pro to see the approach for this follow-up.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.