Which workload most benefits from RadixAttention specifically vs simpler per-request prefix caching?

**RadixAttention** is SGLang's automatic prefix sharing system for the KV cache. When two requests start with the same tokens, the keys and values for that shared start are identical, so they can be computed once and reused. RadixAttention organizes every cached prefix into a radix tree, a compact tree where each path from the root spells out a token sequence. A simpler scheme, like vLLM's hash-based prefix cache, also reuses shared prefixes. The difference is in the branching case. When one base prompt spawns many continuations that diverge and sometimes reconverge, a tree represents that structure naturally. The question asks which workload benefits most. Single-shot unique prompts share nothing. A single streaming user has only one active request at a time, so a plain cache is enough. A batch of unique documents has no overlap either. The agent loop is the winner. It runs the same scaffolding context many times, branches on tool outcomes, explores speculative continuations, then commits or discards them. That branching fan-out is exactly the tree topology RadixAttention encodes, so the shared prefix KV is computed once and reused across every branch.

**RadixAttention** stores all cached prefixes in a radix tree and matches any incoming request against the longest shared path automatically. Its advantage over simpler prefix caching shows up specifically when many requests share a branching prefix. The agent loop is the right answer. A common scaffolding context, the system prompt plus reasoning so far, fans out into multiple branches. Each branch differs only at the tail, after a tool call mixes in. Some branches reconverge. A radix tree represents this fan-out natively. The shared trunk is computed once. Every branch reuses it without recomputing prefill. The other options fail the sharing test. Single-shot unique prompts share nothing across requests. A single streaming user runs one request at a time, which a plain hash-based prefix cache handles. A batch of independent documents has no overlap. The key distinction is that vLLM's hash cache keys on whole-block prefixes and shines on a stable repeated prefix. RadixAttention adds value when the sharing pattern is a tree, with many live branches partially overlapping at once. That is the structure of agents, tree search, and few-shot fan-out.