How should a team disclose MoE model size for honest latency and cost comparisons?

Attempt it

Free · 2 AI evals / day

Quick reference

TL;DR

Disclose both total params (memory) and active params per token (compute), plus top-k and expert count, a single headline number misleads buyers and infra teams alike.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

A MoE model is like a hospital with eight specialist departments but only two treat each patient. Quoting 'eight departments' tells you building size (memory); quoting 'two doctors per patient' tells you wait time (compute). Honest reporting gives both numbers.

Key concepts

Mixture of Experts Inference Optimization

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

6 min: dual-number spec + stakeholder mapping + dishonest reporting traps + model card template.

Real-world examples

Real products, models, and research that use this idea.

DeepSeek-V3 model card reports total and active parameters separately.
Mistral publishes Mixtral as ~47B total with ~13B active inference behavior.
Cloud API pricing for MoE models often references active FLOPs, not total capacity.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow should API pricing reflect MoE dual-number reporting?

Sign in or go Pro to see the approach for this follow-up.

QWhat extra fields matter for expert-parallel multi-GPU deployments?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Publishing only total params for marketing or only active params to hide memory requirements.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.