Same topic, related formats. Practice these next.
RadixAttention wins where many requests share a branching prefix (agent trees, few-shot fan-out) because a radix tree reuses KV across all branches, not just one linear prefix per request.
Imagine a tour guide who memorizes the start of a story so every visitor hears the opening for free. A simple cache helps when one visitor asks the same thing twice. But agents are like a choose-your-own-adventure book: everyone reads the same first chapters, then each picks a different branch, and some branches loop back together. RadixAttention is a smart librarian who maps the whole book as a tree. Every reader who shares chapters reuses the same bookmark, no matter where they branch. So the shared opening is computed once and handed to many branching readers at once. The bigger and bushier the tree of shared beginnings, the more work the librarian saves for everyone.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
3 min: define RadixAttention as a radix-tree KV index, contrast with vLLM block-hash caching, explain why branching agent trees win, and rule out single-shot and single-user cases.
| Aspect | vLLM hash prefix cache | SGLang RadixAttention |
|---|---|---|
| Index structure | Hash of fixed-size token blocks | Radix tree over token sequences |
| Best case | One stable repeated linear prefix | Many branches sharing a common trunk |
| Branching support | Limited; per-block exact match | Native; fork edges where requests diverge |
| Reconvergence | Not represented | Captured as shared nodes |
| Ideal workload | Shared system prompt across users | Agent loops, tree search, few-shot fan-out |
Real products, models, and research that use this idea.
What an interviewer would ask next. Try answering before peeking at the approach.
Red flags and common mistakes that signal junior thinking. Click to expand.
Assuming any prefix caching equals RadixAttention. Single user reuse is handled by a plain hash cache. The radix tree pays off only when many branching requests share overlapping prefixes.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.