Decode

Find the wrong move in 'decode is slow, so let's switch to a smaller FLOP model'

Which quantization combo squeezes the most decode bandwidth per percent of quality lost?

Name the dominant cost driver of GPT-5.5 latency at a 200k-token input

Each decode step loads ___ from HBM, regardless of how many tokens have already been generated

Prefix cache hit lands on a request: does speculative decoding still help that request?

Walk through how CUDA graphs reshape per step decode latency and when the win disappears

TTFT: name the metric it captures and what dominates it

The roofline model in plain language, without the chart, what is it saying?

What unit measures GPU memory bandwidth and why does that number cap decode speed?

End to end latency: fill in the queue, prefill and decode components.

Why do hosted LLM APIs charge separate per million rates for input and output tokens?

Describe what happens during the decode phase of LLM inference.

Match TTFT and TPOT to the latency they measure during streaming

During decode, why is only Q computed for the new token while full K and V come from cache?

Prefill and decode: name the two phases of LLM inference and say which one is compute bound

Decode phase attention is memory bandwidth bound. Explain what flips the regime.

What problem does chunked prefill solve in production LLM serving?

Why does prefill saturate compute while decode is bottlenecked on memory bandwidth?

PremiumWhich statement best explains…

What exactly does the KV cache store and what computational redundancy does it eliminate?

PremiumWhy are output tokens…

PremiumFor a 200-token prompt…

Walk through the byte accounting that proves a single batch decode step is bandwidth bound on H100.

Predict the bandwidth vs compute latency of a single Llama-70B decode step on H100

Spot the errors in this 'API is slow because of network and tokenizer' explanation