Define 'fertility' as a tokenizer metric and explain why high fertility for a language is both a fairness and a model performance problem. · Zenaique

Attempt it

Define the fertility metric for tokenizer evaluation. Explain the two separate problems that high fertility causes: (1) an economic fairness problem for users and (2) a model quality problem in terms of training and inference. Cite the empirical relationship between fertility and downstream accuracy if known.

Free · 2 AI evals / day

Quick reference

TL;DR

Fertility is average tokens per word; high fertility taxes non-English users on cost and context, and starves the model of signal so accuracy lags at fixed compute.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine a vending machine that charges by the piece, and your favorite snack happens to come pre-broken into ten little crumbs while everyone else's comes whole. You pay ten times to get the same snack, and it barely fits in your bag. Worse, when you try to learn what the snack tastes like, you only ever get crumbs, so you never quite get the full flavor. Fertility is how broken-up a language gets. Some languages come whole, others come in crumbs, and the crumb languages get charged more and understood less.

Key concepts

Fertility Token tax

Detailed answer & concept explanation

~4 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

4 min: fertility definition + fairness harm (billing + context) + quality harm (training signal) + 4x-at-2x figure + why both harms are separate + tail-hiding averages + stickiness of the fix.

Real-world examples

Real products, models, and research that use this idea.

OpenAI moved from cl100k_base to o200k_base partly to cut multilingual fertility, lowering both per-token cost and the accuracy gap for non-English GPT-5.5 users.
Google's Gemini 3.1 Pro and Gemma 4 ship large multilingual vocabularies aimed at reducing fertility on Indic, Arabic, and African scripts.
The BLOOM and Aya multilingual projects trained balanced tokenizers specifically to avoid English-centric fragmentation of low-resource languages.
Cohere's enterprise localization work reports fertility alongside accuracy when choosing or extending a tokenizer for a target market.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QFertility is an average. How would you catch a tokenizer that is fine on common words but shreds named entities?

Sign in or go Pro to see the approach for this follow-up.

QYou inherit a deployed model with bad fertility on a target market. What are your options short of full retraining?

Sign in or go Pro to see the approach for this follow-up.

QIs low fertility always good? When might aggressive merging hurt?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating high fertility as a billing inconvenience only, and missing that it also lowers the model's actual accuracy on that language at fixed training compute.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.