Why might a MoE inference API price on active FLOPs instead of total parameter count?
Per-token FLOPs scale with k active experts, not N total, pricing on active compute matches actual inference work.
A MoE model is like a huge toolbox with many specialist tools, but you only use two per job. Billing by total toolbox size would overcharge you for tools you never touched. Pricing on active FLOPs bills for the work actually done, the k experts that ran, not every expert sitting on the shelf.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Why MoE inference APIs price on active FLOPs instead of total parameter count connects architecture understanding to business reality. The MCQ tests whether you grasp that MoE decouples stored capacity from per-token compute, the same insight behind active vs total parameter counting, applied to billing.
Cloud inference providers face a design choice: charge customers based on model size (total params) or based on actual compute consumed (active FLOPs). For dense models, the two are nearly identical, every parameter participates in every forward pass. For MoE models, they diverge sharply. Pricing on the wrong metric either overcharges customers or undercharges providers.
This deep dive explains the compute-memory split, analyzes each MCQ option, and frames how production API economics handle MoE models.
The sections below build from intuition to production practice. Read actively: after each section, pause and restate the key point in your own words, that rehearsal is what converts reading into interview-ready recall.
Why active FLOPs are the right billing unit
Per-token inference cost is dominated by matrix multiplications in attention and FFN layers. In MoE, FFN matmuls run on k of N experts per token, typically k=1 or k=2 while N might be 8, 64, or more.
Active FLOPs capture the matmul volume that actually executes: shared dense layers plus k expert FFNs. Total params count every expert weight in the bank regardless of whether it ran for this token. Billing on total params would charge a Mixtral customer as if 47B parameters computed every forward pass when ~13B actually did.
Active-FLOP pricing aligns marginal cost with marginal resource use, the same principle behind CPU billing by core-hours rather than by total installed RAM.
Enterprise contracts sometimes blend both metrics: a base fee for model capacity (memory reservation) plus per-token active-FLOP charges for usage. Hybrid pricing reflects that providers bear both fixed and marginal costs.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Mistral API pricing reflects MoE active-compute efficiency vs dense models of similar quality.
- DeepSeek publishes active vs total params to clarify cost per token expectations.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhat cost does the provider still bear even with active-FLOP pricing?
Memory for all experts, expert-parallel sharding infrastructure, idle expert capacity on GPUs.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Assuming total parameter count equals per-token compute cost in MoE serving.
60 second bullets to scan on the way to the call.
Active FLOPs scale with k, not N
Total params determine memory footprint
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.