Quantization

Deploying a vision-language model on a phone or AR headset instead of a server: what changes?

Match each vector compression scheme to its core mechanism.

Match each GPU generation to the quantization format that its tensor cores run natively.

Which quantization combo squeezes the most decode bandwidth per percent of quality lost?

Rank these chat API cost levers from highest to lowest typical ROI

Match Hopper FP8 and Blackwell FP4 tensor cores to their throughput and bandwidth effects on decode

In weight quantization, what changes and what stays the same?

Where does FP8 fit in modern LLM serving and which GPUs support it?

Match each 4-bit format to its niche in inference and fine-tuning.

Define activation quantization and explain how W8A8 differs from weight only quant.

What subtle problem do you hit when merging a LoRA back into an NF4 quantized base, and why?

What is the structural difference between FP4 and NF4?

INT8 KV cache vs INT8 weight quantization, which one is easier in production?

Name two attention side strategies the runtime can apply when KV memory runs short mid request.

How does QLoRA fit 65B fine-tuning onto a single 48GB GPU?

Which statements about QLoRA are true?

PremiumWhy does QLoRA use…

PremiumWhy is 'LoRA is…

Which statements about FP8, INT8 and INT4 weight quantization are correct?

Match each weight quantization regime to what it buys you and where it breaks

PremiumWhat does W8A8 vs…

Match each weight/activation quant regime to its mechanism and best fit workload

Spot the errors in this tensor core utilization claim

On the roofline, which intervention moves an LLM decode workload most directly toward higher peak throughput?

Match each post-training quantization method to what it protects against