What is RadixAttention in SGLang and how does it differ from vLLM's prefix sharing?
Describe RadixAttention as implemented in SGLang. Be specific about the data structure used, how prefix matches are detected across requests, and why this matters for agent and few-shot workloads.
RadixAttention indexes cached KV blocks in a persistent radix tree keyed by token prefix, so a new request automatically reuses the longest matching prefix and prefills only its divergent suffix.
Imagine a library where every book that starts with the same chapter shares one physical copy of that chapter, branching into separate copies only when the stories diverge. A radix tree is that shared shelf for token sequences. When a new reader arrives with a story, a clerk walks the shelf word by word, follows the path that matches, and hands over every shared chapter already on hand. The reader only has to write the new part their story adds. Because so many readers begin with the same long preface, the system rarely rewrites that preface twice. The walk takes about as long as the shared part, not as long as the whole library, so finding the match is almost free.
Detailed answer & concept explanation~8 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
3 min: radix tree keyed by token prefix + O(prefix) tree-walk match + reference counting and LRU eviction + vLLM hashing contrast + cache-aware scheduling + agent and few-shot wins.
Real products, models, and research that use this idea.
- SGLang ships RadixAttention as its default KV cache manager, serving Llama 4 and Qwen 3 with automatic cross-request prefix sharing.
- Agent frameworks running long fixed system prompts and tool scaffolding on SGLang see large hit-rate gains because every loop step reuses the same prefix.
- Few-shot pipelines that prepend one long shared exemplar block reuse that entire block across thousands of requests via a single tree path.
- Tree-of-thought and branched sampling workloads fork from a shared context node, so RadixAttention shares the trunk and branches only the divergent suffixes.
- vLLM added automatic prefix caching with hash-based block matching, the closest comparison point interviewers expect you to contrast against.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy is reference counting necessary on radix tree nodes rather than plain LRU alone?
QHow does cache-aware scheduling change the win compared to the tree alone?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Saying RadixAttention is just vLLM prefix sharing with a new name. The defining difference is the persistent automatic cross-request radix tree versus per request hash lookups.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.