Identify PagedAttention and the project it ships with

**PagedAttention** is a memory-allocation trick for the KV cache, introduced by the vLLM serving engine from UC Berkeley in 2023. Before PagedAttention, serving stacks allocated the KV cache as one contiguous slab per sequence, sized for the maximum context the model could handle. If the model supported 8k tokens but the user only sent a 200-token request, the allocator still reserved 8k worth of cache. That reservation sat half-empty while other waiting requests could not be admitted because HBM was nominally full. PagedAttention borrows the operating-system paging idea. The KV cache is split into fixed-size blocks (typically 16 tokens each), and each sequence holds a block table that lists which physical blocks it currently uses. When the sequence grows by one token, it asks for another block from a free pool. When it finishes, its blocks return to the pool for reuse. The consequence is dense HBM packing. Many more concurrent sequences fit on the same GPU, batches stay full, and throughput jumps. That was the single biggest reason vLLM outperformed earlier servers when it launched. The attention kernel itself does not change; only the memory layout and the address indirection do.

**PagedAttention** is the block-based KV-cache allocator that ships with vLLM, the open-source LLM serving engine from UC Berkeley. It is the canonical example of borrowing a systems idea (OS virtual memory paging) and applying it to ML serving. The problem it solves is fragmentation. Without paging, the KV cache must be allocated contiguously per sequence. The only safe size for that contiguous allocation is the model's maximum supported context length, because the system does not know in advance how long the sequence will become. So a 200-token request reserves space for 8k or 32k tokens. Across many concurrent sequences, this wastes tens of percent of HBM to internal fragmentation. The GPU looks full but is mostly holding placeholders. PagedAttention divides the KV cache into fixed-size blocks (typically 16 tokens). Each sequence maintains a block table mapping its logical positions to physical block indices, exactly like a virtual-memory page table. As the sequence decodes, it claims more blocks on demand from a global free pool. As sequences finish, their blocks return to the pool. The benefits compound: more concurrent sequences fit, larger batches stay full, prefix caching becomes natural (two sequences sharing a prefix can share blocks), and continuous batching has somewhere to put newly admitted requests mid-flight. PagedAttention is the foundation under almost every modern serving optimization.