Match open frontier MoE models to a distinguishing trait
Drag each answer to line up with its matching prompt
Mixtral 8x7B
8 routed experts, top-2 active, 7B class FFN per expert
DeepSeek-V3
Meta's open MoE variant with high expert count and conditional routing
Llama 4 Maverick
Fine-grained experts with shared always on FFNs and aux loss free bias balancing
Qwen3 MoE
Alibaba open MoE with configurable expert count and top-k routing
Open frontier MoEs differ: Mixtral 8×7B top-2, DeepSeek fine-grained + shared + bias balancing, Llama 4 Maverick high-count MoE, Qwen3 configurable top-k.
Like different car makers' hybrid engines, all use the same basic idea (only some cylinders fire at once) but Mixtral has 8 big pistons firing 2 at a time, DeepSeek uses many smaller pistons plus an always-on engine and a smart traffic light, and others tune their own counts and routing rules.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Open frontier MoE models multiplied in 2024-2026, and interviewers expect landmark recognition, not every hyperparameter, but the one distinguishing trait per model that signals you follow the field. This easy match question pairs four names every candidate should know with configs that differentiate them.
The trap is treating MoE as one pattern. Mixtral taught the world 8×7B naming; DeepSeek taught fine-grained + shared + aux loss free; Meta and Alibaba shipped competitive open tiers with their own expert counts. Knowing which is which prevents embarrassing conflation in system design discussions.
This deep dive gives enough depth on each pair to defend your matches in follow-up questions, especially the Mixtral active-params misconception.
Easy match questions still gate credibility. Confusing Mixtral's 8×7B naming with DeepSeek's fine-grained recipe signals you have not opened a model card since 2023. The four pairs here are landmark anchors, one sentence each should be reflexive before you touch hyperparameter trivia.
Easy does not mean skip depth on follow-ups, Mixtral active-params and DeepSeek shared experts are the two most common traps immediately after matching.
Mixtral 8x7B: the naming landmark
Mixtral 8x7B means 8 expert FFNs, each approximately 7B-parameter scale, with top-2 active per token. Total expert-parameter pool is large; active FFN compute per token reflects two experts, not eight, roughly 12-13B-class FFN work plus attention and other shared layers.
This is the most common MoE misconception in interviews: '8×7=56B active.' Wrong. The × counts experts in the bank; top-k counts how many fire.
Mixtral uses token-choice routing and Switch-style auxiliary load balancing heritage. It is the reference answer when explaining MoE to someone who only knows dense 7B/70B naming.
Mixtral active-params trap: interviewers love asking "how many parameters fire per token?" after you match 8×7B. Answer top-2 of eight experts → ~12-13B active FFN class, not 56B. The match pair is only the first move in a sizing conversation.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Mistral AI released Mixtral 8x7B and 8x22B as open-weight MoE models with top-2 routing.
- DeepSeek-V2/V3 open weights demonstrate fine-grained MoE with shared experts and aux loss free balancing.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you compare serving cost of Mixtral vs DeepSeek at similar quality?
Compare active FFN FLOPs, total expert memory footprint, expert count impact on HBM, and quantization layout, not headline total params alone.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Thinking Mixtral 8x7B activates 56B parameters per token, only top-2 of 8 experts run (~12-13B active FFN class).
60 second bullets to scan on the way to the call.
Mixtral 8x7B naming and top-2
Active vs total params for Mixtral
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.