Why does DeepSeek-V3 cap each token's experts to a small set of nodes?

In a large MoE like **DeepSeek-V3**, hundreds of experts are spread across many GPUs, organized into nodes (typically 8 GPUs per node). Within a node, GPUs talk over NVLink at hundreds of GB/s. Between nodes, they talk over InfiniBand or RoCE at tens of GB/s. The intra-node link is roughly an order of magnitude faster. DeepSeek-V3 routes each token to about 8 experts. If those 8 experts can be anywhere in the cluster, the token has to traverse the slow inter-node fabric for almost all of them. Across the whole batch, this saturates the slow links and step time stalls. **Node-limited routing** is a rule baked into the gate: when picking experts for a token, you may only pick from at most M nodes (DeepSeek-V3 uses M=4). That bounds the inter-node fan-out per token. Most dispatch stays on NVLink, and only the bounded slice goes over the slow fabric. It is a systems constraint, not a quality or memory trick.

**Node-limited routing** is a hardware-aware constraint on the routing rule. The point is to keep all to all communication concentrated on the fast intra-node interconnect. The setup. In an expert-parallel cluster, experts are pinned to GPUs and GPUs are grouped into nodes. Intra-node bandwidth (NVLink, NVSwitch) is roughly 5 to 10 times faster than inter-node bandwidth (InfiniBand, RoCE). When a token routes to k experts spread randomly across N nodes, the all to all step has to ship that token's hidden state to all the right nodes, do the FFN computation, and ship results back. Without the constraint, with k=8 and many nodes, almost every token crosses the slow fabric. The cross-node all to all becomes the step bottleneck. The constraint. DeepSeek-V3 restricts each token's k selected experts to lie in at most M=4 nodes out of the full cluster. The gate score first picks the top M nodes (by sum of top-2 expert affinities per node), then picks the top k experts within those nodes. The math now puts at most M nodes per token in the all to all pattern, so cross-node volume is bounded by M times batch size, not k times batch size, and most dispatch stays on NVLink. It is not about memory (experts are still pinned to whichever GPU holds them), not about specialization (specialization comes from training dynamics), and not about determinism (token routing is still data-dependent).