According to tokenizer fairness research, if a language's fertility doubles, what typically happens to downstream model accuracy? · Zenaique

Attempt it

Quick reference

TL;DR

Higher fertility predicts lower downstream accuracy: doubling tokens per word correlates with roughly 4x more training cost to reach the same quality, so under-trained languages lag.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine two students copying the same story by hand. One writes in shorthand and finishes in 100 strokes; the other has to spell everything letter by letter and needs 400 strokes. Given the same amount of writing time, the shorthand student rereads and learns the story far better. A tokenizer is the shorthand. Languages that get chopped into many tiny tokens are like the slow writer: the model spends its limited budget on mechanical pieces instead of meaning, so it understands those languages less well.

Key concepts

Fertility Token tax

Detailed answer & concept explanation

~4 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

2 min: fertility definition + why direction is degrade not improve + 4x-at-2x figure + training-signal mechanism + context-window half + the two wrong distractors.

Real-world examples

Real products, models, and research that use this idea.

OpenAI's o200k_base improved multilingual fertility over cl100k_base, narrowing the per-token cost and quality gap for non-English GPT-5.5 users.
Google's Gemma 4 and Gemini 3.1 Pro use large multilingual vocabularies specifically to lower fertility on Indic and African scripts.
The Aya and BLOOM multilingual efforts trained balanced tokenizers to avoid the fragmentation that English-centric vocabularies impose on low-resource languages.
Cohere's multilingual embedding and command models report fertility alongside accuracy when comparing tokenizer choices for enterprise localization.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QIf fertility predicts accuracy, can you just retrain the tokenizer on a balanced corpus and swap it into an existing model?

Sign in or go Pro to see the approach for this follow-up.

QFertility is an average. How would you measure fairness for a language with both common and rare words?

Sign in or go Pro to see the approach for this follow-up.

QTwo languages have equal fertility but different accuracy. What else is driving the gap?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Assuming more tokens per word gives the model finer-grained signal and therefore better accuracy, when the empirical relationship runs the opposite way.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.