Find the active-parameter mistake in this Mixtral 8×7B claim.

Attempt it

Click any words you think contain an error. Click again to unmark.

Mark at least one word to submit.

Quick reference

TL;DR

Mixtral 8×7B activates top-2 of eight experts per token (~12–13B active), not 56B. Total expert capacity is not per-token compute.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine a hospital with eight specialist doctors, each as skilled as a 7B-class expert. Mixtral does not send every patient to all eight doctors at once. A receptionist picks the two best specialists for each case. The work per patient is two doctors plus shared staff, not eight times one doctor. The name 8×7B describes how many specialists exist on staff, not how many treat each patient.

Key concepts

Mixture of Experts Inference Optimization

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

6 min: naming convention + top-k arithmetic + memory vs compute + interview quoting rules.

Real-world examples

Real products, models, and research that use this idea.

Mistral's Mixtral 8×7B highlights ~47B total params while inference behaves like a ~13B active model.
DeepSeek-V2 reports total vs active params separately for cost per token comparisons.
Cloud GPU provisioning for MoE serving budgets memory for all experts, not just top-k.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would active param count change if Mixtral used top-1 instead of top-2?

Sign in or go Pro to see the approach for this follow-up.

QWhy do API providers sometimes price MoE on active FLOPs?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Multiplying 8 × 7B and calling that the active parameter count per forward pass.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.