An MoE layer has 8 experts and uses top-2 routing. The load balancer is working perfectly, so routed traffic is spread evenly. A batch of tokens flows through the layer. What fraction of all tokens does each individual expert process?
Top-2 routing gives every token two expert assignments, so a perfectly balanced 8-expert layer sends 2/8 = 25% of tokens to each expert, not 1/8.
Imagine a school cafeteria with 8 food counters where every student must visit exactly 2 of them: one for a main dish, one for a side. If 100 students come through at lunch, the counters together handle 200 visits, because each student counts twice. A perfect line monitor spreads those 200 visits evenly, so each counter serves 25 students, a quarter of the whole school. An MoE layer works the same way. Each token is the student, the experts are the counters, and top-2 routing is the rule that everyone visits two stations. The tempting wrong answer, one eighth, comes from dividing 100 students across 8 counters and forgetting that every student stands in two lines.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
On paper this question is one division. In practice, the number it produces is the foundation of every MoE capacity plan, utilization dashboard, and alert threshold, and it is missed surprisingly often because the intuitive answer is wrong by a factor of two.
The intuition trap works like this: 8 experts, traffic spread evenly, so each expert must see 1/8 of tokens. That reasoning quietly assumes each token goes to exactly one expert. Top-2 routing breaks the assumption: the router sends every token to two experts, so the layer's workload is twice the token count, and the even split lands at 25% per expert.
This deep dive walks through the counting argument, then shows where the same term reappears in capacity buffers, monitoring baselines, and the configurations of frontier MoE models. By the end, the 2/8 arithmetic should feel less like a puzzle and more like the first line of a serving runbook.
Count assignments, not tokens
The clean way to reason about any routed MoE layer is to count expert assignments rather than tokens. A batch of T tokens under top-k routing produces exactly T times k assignments, because the router picks k distinct experts for every token. The token count never changes; the workload does.
With T = 100 and k = 2, the layer creates 200 assignments. Perfect balance means those 200 assignments are spread evenly over the N = 8 experts, so each expert receives 25 assignments. Since each assignment corresponds to one token visiting that expert, each expert processes 25 of the 100 tokens.
Written as a general identity:
The T cancels, which is the point: the balanced share does not depend on batch size at all. It is a pure property of the routing configuration. Top-2 of 8 gives 25%. Top-1 of 8, the Switch Transformer setting, gives 12.5%. Top-8 of 256, the DeepSeek-V3 setting, gives about 3.1%.
Notice what the perfect load balancer in the question is doing for you. It does not change how many assignments exist. It only guarantees the even spread, which is exactly the condition that lets you divide total assignments by N without worrying about skew.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Mixtral 8x7B routes each token to 2 of 8 experts, so balanced utilization targets 25% of tokens per expert.
- DBRX selects 4 of 16 fine-grained experts, which lands on the same 25% balanced share with finer specialization.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat does each expert's share become with 64 experts and top-8 routing?
Apply the same assignment counting, then ask why fine-grained designs prefer many small shares.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Answering 1/8 by dividing tokens across 8 experts, which forgets that top-2 routing assigns every token to two experts and doubles the total assignments.
60 second bullets to scan on the way to the call.
Expected per-expert share formula under balanced routing
Why total assignments are tokens times k, not tokens
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.