What is the primary mechanism by which PagedAttention increases serving throughput?

Attempt it

Quick reference

TL;DR

PagedAttention stores the KV cache in fixed-size blocks tracked by a block table, killing the fragmentation that contiguous reservation wastes, so more requests fit and batch size rises.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine a parking lot where every car must park in one long unbroken strip of spaces, and you reserve the strip based on the longest trip anyone might take. Most drivers leave early, so huge stretches sit empty but blocked. The lot looks full while half of it is wasted. PagedAttention instead breaks the lot into small numbered bays and hands out one bay at a time, only as each car actually needs it. A little map records which bays belong to which car, so they can be scattered anywhere. Nothing is reserved that is not used, the empty gaps vanish, and far more cars fit at once. Fitting more cars at once is exactly what raises a serving system's throughput.

Key concepts

PagedAttention KV cache Memory fragmentation Continuous batching

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

3 min: fragmentation in contiguous KV reservation, block table and paged allocation, why freed memory raises batch size, and how decode bandwidth amortization turns that into throughput.

Real-world examples

Real products, models, and research that use this idea.

vLLM (UC Berkeley) introduced PagedAttention in 2023 and is the default serving engine behind most open-source LLM deployments in 2026.
SGLang builds on paged blocks with RadixAttention to share prompt prefixes across batched requests serving Llama 4 and Qwen 3.
TensorRT-LLM (NVIDIA) ships a paged KV cache plus chunked prefill as the production reference on H100 and B200 GPUs.
Anthropic and OpenAI prompt caching exposes the same prefix-sharing idea as a billing feature, charging a fraction for repeated prompt prefixes.
Hugging Face TGI adopted paged KV allocation to lift concurrent-request throughput for hosted Llama 4 and Mistral Large 3 endpoints.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhy does eliminating fragmentation translate into higher throughput rather than just lower memory use?

Sign in or go Pro to see the approach for this follow-up.

QHow do copy-on-write blocks help beam search and parallel sampling?

Sign in or go Pro to see the approach for this follow-up.

QDoes PagedAttention compete with or compose with GQA and MLA?

Sign in or go Pro to see the approach for this follow-up.

QWhat overhead does the block table and non-contiguous layout introduce?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Saying PagedAttention speeds up the attention math or compresses the KV cache. It changes only the memory layout; the arithmetic per token is unchanged.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.