Same topic, related formats. Practice these next.
Disaggregation pays off only when prefills are both large and frequent enough to stall decodes: long prompts at high concurrency, the RAG and agent regime.
Picture one chef doing two very different jobs. Sometimes a huge catering order arrives and takes all his attention for several minutes. Meanwhile, dozens of customers want one bite-sized appetizer every few seconds, served smoothly. If the chef stops to cook the catering order, every appetizer customer waits and gets annoyed. The fix is two kitchens: one chef handles big catering bursts, the other keeps the steady trickle of appetizers flowing. This only helps if big orders are both large and arrive often. If catering orders are tiny, or almost never arrive, splitting the kitchens just adds overhead and a handoff cost between them.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: prefill vs decode bottlenecks, the two-part interference test, why each wrong trace fails it, the KV cache transfer cost, and disaggregation vs chunked prefill.
Real products, models, and research that use this idea.
What an interviewer would ask next. Try answering before peeking at the approach.
Red flags and common mistakes that signal junior thinking. Click to expand.
Picking the long-completion creative-writing trace. That workload is almost all decode with trivial prefill, so there is nothing to disaggregate; the collision that disaggregation removes never happens.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.