Recognize vLLM and the optimization that made it famous

**vLLM** is the open-source LLM serving engine originally built at UC Berkeley in 2023. The name's 'v' nods to the Berkeley systems lineage; the rest of the acronym just says LLM. It is one of the three serving stacks every modern inference engineer is expected to know (the other two are TGI and TensorRT-LLM). What made vLLM famous is a single optimization called **PagedAttention**. Before vLLM, serving stacks allocated the KV cache as one contiguous slab per sequence, sized for the model's maximum context length. Real traffic has variable sequence lengths, so most of those reservations sat empty. Fragmentation was costing tens of percent of HBM. PagedAttention borrowed the operating-system virtual-memory paging idea. It split the KV cache into fixed-size blocks (typically 16 tokens), gave each sequence a block table that pointed to its blocks, and admitted new sequences from a shared free pool. HBM packed densely, batches stayed full, and reported throughput jumped roughly an order of magnitude on real workloads. vLLM also ships continuous batching, prefix caching, and FP8 quantization, but PagedAttention is the signature feature and the answer to 'why is vLLM fast?'

**vLLM** is the open-source LLM serving engine that came out of UC Berkeley in 2023. It became the default open-source choice for production LLM serving almost immediately on release. The single optimization vLLM is associated with is **PagedAttention**. The framing in the launch paper was direct: existing serving stacks were wasting 60 to 80 percent of HBM to KV-cache fragmentation. Their contiguous-allocation strategy forced every sequence to reserve max-context bytes up front, even if the sequence was short. PagedAttention fixed this by carving the KV cache into fixed-size blocks (typically 16 tokens), giving each sequence a block table mapping logical positions to physical block indices, and allocating blocks on demand from a global free pool. The mechanic is direct OS virtual-memory paging applied to attention's KV cache. The attention math is unchanged; only the addressing changes. The downstream effect is enormous: HBM packs densely, many more concurrent sequences fit per GPU, continuous batching becomes practical because new requests can be admitted mid-flight, and prefix caching becomes natural through reference-counted shared blocks. Reported throughput on benchmark workloads leaped roughly an order of magnitude versus earlier servers. PagedAttention also became influential beyond vLLM: TGI, TensorRT-LLM (as 'in-flight batching'), and SGLang (as RadixAttention) all implement variations of the same idea. Continuous batching, prefix caching, and FP8 quantization are also vLLM features, but they were enabled by PagedAttention and remain downstream of it.