An MoE layer has 8 experts, top-1 routing, and capacity factor 1.0. A batch of 4096 tokens arrives, but routing is skewed: expert 0 is chosen by 50% of tokens, while the remaining tokens spread evenly across experts 1-7. What fraction of the batch gets dropped, and how many tokens is that?
The answer is 1536 of 4096 tokens, which is **37.5 percent**.
Start with **per-expert capacity**. The formula for top-1 routing is `CF x T x k / N`, where T is batch tokens, k is top-k, N is expert count. Plug in CF=1.0, T=4096, k=1, N=8 and you get 512 slots per expert.
Now apply the routing distribution. Expert 0 is hot at 50%, so it receives 0.5 x 4096 = 2048 tokens. Its buffer only holds 512. The overflow is 2048 - 512 = 1536 tokens, which means 1536 tokens cannot enter expert 0's FFN.
The remaining 2048 tokens spread evenly across experts 1 through 7. That gives each cold expert about 2048 / 7, which is 292.6 (call it 293) tokens. Each cold expert has 512 slots, so they are well under capacity. They drop nothing.
Total drops = 1536. As a fraction of the batch: 1536 / 4096 = 0.375 = **37.5 percent**.
The second-order observation worth noting: the cold experts collectively waste 7 x (512 - 293) = 1533 slots. So the hot expert is dropping 1536 tokens while the rest of the layer has 1533 unused slots. This is the classic signature of routing skew, you cannot fix it by raising capacity factor because the slots already exist, they are just on the wrong GPUs.
**Answer: 37.5 percent of the batch (1536 of 4096 tokens) is dropped.**
**Step 1: per-expert capacity.** With top-1 routing, every batch token goes to exactly one expert, so the total dispatched count equals the batch size T. Distributing T uniformly across N experts gives `T / N` tokens per expert at balanced demand. Capacity factor scales that:
$$\text{capacity}_e = \text{CF} \cdot \frac{T \cdot k}{N} = 1.0 \cdot \frac{4096 \cdot 1}{8} = 512$$
**Step 2: tokens received per expert.** Skew sends 50 percent to expert 0:
- Expert 0 receives 0.5 x 4096 = 2048 tokens.
- Experts 1 through 7 split the remaining 2048 tokens evenly: 2048 / 7 ~ 293 tokens each.
**Step 3: per-expert drops.**
- Expert 0: receives 2048, capacity 512, drops 2048 - 512 = **1536 tokens**.
- Experts 1-7: receive 293 each, capacity 512, drops 0.
**Step 4: total drop fraction.** Total drops = 1536. As a fraction: 1536 / 4096 = 0.375 = **37.5 percent**.
**The asymmetric waste.** Beyond the answer, the structure of the problem deserves attention. The cold experts together waste 7 x (512 - 293) ~ 1533 unused slots. The hot expert drops 1536 tokens. The total slots in the layer (8 x 512 = 4096) exactly match the batch size, but the distribution mismatch turns roughly 1.5K slots into dead space while another 1.5K tokens get skipped. Raising the capacity factor cannot recover the cold-expert slack because the FFN dispatch is per-expert, not pooled. This is why capacity-factor headroom helps (CF=2.0 would have absorbed the skew) but expert replication or rebalanced routing is the real cure.
Quick reference
TL;DR
Capacity is 512 per expert; expert 0 receives 2048 and drops 1536, the other seven are under capacity, so 1536 of 4096 (37.5%) are silently dropped through the residual.
Picture eight ticket counters at a stadium where each counter can serve exactly 512 fans per minute. Suddenly half the crowd swarms one counter: 2048 fans want counter zero, but only 512 fit. The other 1536 fans waiting at counter zero do not get a ticket at all, they just walk past the counter and into the game with an unstamped hand. Meanwhile counters one through seven serve only about 293 fans each and have 219 idle slots going to waste. Out of 4096 fans total, 1536 missed their ticket entirely, which is 37.5 percent. The stadium logs only count tickets sold per counter, so nobody notices the unstamped fans until much later when they all come back to complain about the experience.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Dropless MoE serving (MegaBlocks) Block-sparse dispatch eliminates the fixed-capacity model entirely; this arithmetic exercise becomes irrelevant because no tokens are dropped by design.
Top-k > 1 routing with the same skew analysis Higher k changes the capacity formula (multiplies by k) and changes how cold-expert slack distributes; the same intuition applies but the numbers shift.
Expert-choice routing instead of token-choice Expert-choice guarantees balanced load by construction (every expert picks the same number of tokens), so capacity-driven drops cannot happen the same way; capacity factor in that regime governs different things.
Brief transient skew on otherwise balanced traffic A spike that subsides in seconds may be tolerable if the per-step drops average out; the question's setup assumes sustained skew where drops accumulate across requests.
Quick scan · 2–4 min
Everything important, quickly.
8 min: derive capacity per expert, apply the skewed distribution, compute per-expert drops, sum to the total fraction, then discuss the asymmetric-waste structure, the alternatives to higher capacity factor, and how production stacks size CF for observed skew.
Real-world examples
Real products, models, and research that use this idea.
1Switch Transformer's original ablations include exactly this style of capacity versus skew analysis to motivate the aux loss formulation.
2Mixtral 8x22B deployments on vLLM ship with a default capacity factor near 1.5 precisely to absorb this magnitude of skew without drops.
3DeepSeek-V3's aux loss-free balancing reports per-expert token counts in the technical report, showing skew metrics analogous to this exercise.
4MegaBlocks dropless MoE was developed to eliminate this exact arithmetic from production serving by replacing fixed buffers with block-sparse dispatch.
5Qwen-MoE serving guides include sample calculations of capacity versus observed skew to help operators tune CF for their workload.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat capacity factor would push total drops below 1 percent in this same scenario?
Sign in or go Pro to see the approach for this follow-up.
QHow does the answer change if top-k is 2 instead of 1, keeping the same skew structure?
Sign in or go Pro to see the approach for this follow-up.
QWhy does the cold-expert slack not help the hot expert mathematically?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Computing 50% as the drop rate (treating all hot-expert traffic as dropped) or forgetting that cold experts under capacity drop nothing; the right answer accounts only for the overflow at the hot expert divided by total batch.