Predict the dropped token fraction when one expert attracts half the batch at capacity factor 1.0.

Predict output·Hard·4.0 · 0·~2 min·Asked atEy Figure Ai Lakera

Attempt it

An MoE layer has 8 experts, top-1 routing, and capacity factor 1.0. A batch of 4096 tokens arrives, but routing is skewed: expert 0 is chosen by 50% of tokens, while the remaining tokens spread evenly across experts 1-7. What fraction of the batch gets dropped, and how many tokens is that?

The answer is 1536 of 4096 tokens, which is **37.5 percent**. Start with **per-expert capacity**. The formula for top-1 routing is `CF x T x k / N`, where T is batch tokens, k is top-k, N is expert count. Plug in CF=1.0, T=4096, k=1, N=8 and you get 512 slots per expert. Now apply the routing distribution. Expert 0 is hot at 50%, so it receives 0.5 x 4096 = 2048 tokens. Its buffer only holds 512. The overflow is 2048 - 512 = 1536 tokens, which means 1536 tokens cannot enter expert 0's FFN. The remaining 2048 tokens spread evenly across experts 1 through 7. That gives each cold expert about 2048 / 7, which is 292.6 (call it 293) tokens. Each cold expert has 512 slots, so they are well under capacity. They drop nothing. Total drops = 1536. As a fraction of the batch: 1536 / 4096 = 0.375 = **37.5 percent**. The second-order observation worth noting: the cold experts collectively waste 7 x (512 - 293) = 1533 slots. So the hot expert is dropping 1536 tokens while the rest of the layer has 1533 unused slots. This is the classic signature of routing skew, you cannot fix it by raising capacity factor because the slots already exist, they are just on the wrong GPUs.

**Answer: 37.5 percent of the batch (1536 of 4096 tokens) is dropped.** **Step 1: per-expert capacity.** With top-1 routing, every batch token goes to exactly one expert, so the total dispatched count equals the batch size T. Distributing T uniformly across N experts gives `T / N` tokens per expert at balanced demand. Capacity factor scales that: $$\text{capacity}_e = \text{CF} \cdot \frac{T \cdot k}{N} = 1.0 \cdot \frac{4096 \cdot 1}{8} = 512$$ **Step 2: tokens received per expert.** Skew sends 50 percent to expert 0: - Expert 0 receives 0.5 x 4096 = 2048 tokens. - Experts 1 through 7 split the remaining 2048 tokens evenly: 2048 / 7 ~ 293 tokens each. **Step 3: per-expert drops.** - Expert 0: receives 2048, capacity 512, drops 2048 - 512 = **1536 tokens**. - Experts 1-7: receive 293 each, capacity 512, drops 0. **Step 4: total drop fraction.** Total drops = 1536. As a fraction: 1536 / 4096 = 0.375 = **37.5 percent**. **The asymmetric waste.** Beyond the answer, the structure of the problem deserves attention. The cold experts together waste 7 x (512 - 293) ~ 1533 unused slots. The hot expert drops 1536 tokens. The total slots in the layer (8 x 512 = 4096) exactly match the batch size, but the distribution mismatch turns roughly 1.5K slots into dead space while another 1.5K tokens get skipped. Raising the capacity factor cannot recover the cold-expert slack because the FFN dispatch is per-expert, not pooled. This is why capacity-factor headroom helps (CF=2.0 would have absorbed the skew) but expert replication or rebalanced routing is the real cure.

TL;DR

Capacity is 512 per expert; expert 0 receives 2048 and drops 1536, the other seven are under capacity, so 1536 of 4096 (37.5%) are silently dropped through the residual.

Memory aid

Easy to grasp

Picture eight ticket counters at a stadium where each counter can serve exactly 512 fans per minute. Suddenly half the crowd swarms one counter: 2048 fans want counter zero, but only 512 fit. The other 1536 fans waiting at counter zero do not get a ticket at all, they just walk past the counter and into the game with an unstamped hand. Meanwhile counters one through seven serve only about 293 fans each and have 219 idle slots going to waste. Out of 4096 fans total, 1536 missed their ticket entirely, which is 37.5 percent. The stadium logs only count tickets sold per counter, so nobody notices the unstamped fans until much later when they all come back to complain about the experience.

Key concepts

Mixture of Experts

Concept explanation~2 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

This question rewards a specific kind of senior-level discipline: the ability to compute the answer cleanly while also seeing the structural lesson hiding inside the arithmetic. The number 37.5 percent is the easy part. The hard part is seeing why this calculation maps directly to how production stacks size capacity factor, why expert replication is preferred over raising CF, and how routing skew compounds across layers.

The walkthrough below builds the answer step by step, then steps back to extract the production-systems lesson. A candidate who delivers only the numeric answer is at the bar; one who connects the arithmetic to placement strategy, capacity tuning, and downstream quality compounding is well past it.

The senior framing: the answer is 1536 / 4096 = 37.5 percent. The lesson is that the layer simultaneously drops 1536 tokens and wastes 1533 slots, so raising capacity factor pays globally to fix a local imbalance. Expert replication or routing rebalancing is the real cure.

Deriving the per-expert capacity

The formula

The per-expert capacity in top-k routing is:

\text{capacity}_e = \lceil \text{CF} \cdot \frac{T \cdot k}{N} \rceil

T is batch tokens, k is top-k, N is expert count, CF is capacity factor. The numerator T x k is the total number of token-expert dispatch events: in top-1, each token dispatches once, so total events equal T; in top-2, each token dispatches twice, so total events equal 2T. Dividing by N gives the per-expert share at perfectly balanced demand.

Plugging in the question's values

With T=4096, k=1, N=8, CF=1.0:

\text{capacity}_e = 1.0 \cdot \frac{4096 \cdot 1}{8} = 512

Every expert has 512 slots. There are 8 experts, so the layer has 8 x 512 = 4096 total slots, exactly equal to the batch size. This is what CF=1.0 means: enough capacity for perfectly balanced traffic, zero headroom for skew.

Why CF=1.0 is the brittle baseline

The practical implication is that any deviation from uniform routing immediately produces drops at the hot expert, with no compensating absorption elsewhere. The cold experts have exactly enough slots for their balanced share and not one more; they cannot help the hot expert even mathematically, because dispatch is per-expert, not pooled.

This is why production stacks set CF above 1.0. Common values are 1.25 (light skew), 1.5 (typical production), 2.0 (heavily skewed workloads). The exact value should be measured per workload.

Applying the skew and computing drops

The asymmetric-waste structure

Production implications and downstream compounding

Situations where this technique stops working.

2–4 min · Everything important, quickly.

Real products, models, and research that use this idea.

Switch Transformer's original ablations include exactly this style of capacity versus skew analysis to motivate the aux loss formulation.
Mixtral 8x22B deployments on vLLM ship with a default capacity factor near 1.5 precisely to absorb this magnitude of skew without drops.

What an interviewer would ask next. Try answering before peeking at the approach.

QWhat capacity factor would push total drops below 1 percent in this same scenario?

Expert 0 needs at least 2048 / 0.99 ~ 2069 capacity to drop fewer than 1 percent of its own load. Per-expert capacity is CF x 512, so CF needs to be ~ 4.04. Note the brutal cost: a 4x global capacity factor to handle one hot expert, which is why replication is preferred.

2 more follow-ups an interviewer would ask next. Sign in to reveal them.

Red flags & common mistakes

The phrases that signal junior thinking. Click to expand.

Most common mistake

Computing 50% as the drop rate (treating all hot-expert traffic as dropped) or forgetting that cold experts under capacity drop nothing; the right answer accounts only for the overflow at the hot expert divided by total batch.

60 second bullets to scan on the way to the call.

Per-expert capacity formula and what each term means
Why drops happen only at experts exceeding capacity

Primary sources. Browse if you want the original framing.