What is STRR and how does it complement fertility as a tokenizer fairness metric?
STRR is the fraction of a language's vocabulary that maps to single tokens; it measures coverage breadth across the word space, catching the rare-word fragmentation that a frequency-weighted fertility average hides.
Imagine grading a grocery store on how easy it is to grab what you need in one reach. Fertility is like timing an average shopper, who mostly buys the same ten popular items kept right at the front, so the store looks fast. STRR instead walks every aisle and counts how many of all the products are within easy reach. A store can look fast for the regulars while burying everything else on high shelves. STRR is the full-aisle audit; fertility is the regular-shopper stopwatch. You want both to know who the store really serves.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Tokenizer fairness debates usually start and stop with fertility, the average tokens per word, because it maps directly to billing. But a single average is easy to game, and a tokenizer that looks fair on the average can be quietly brutal on the words that matter most. That is the gap STRR was introduced to close.
The whole point of this question is to show you understand that fertility and STRR are different statistics of the same tokenization, not two names for the same thing. One is a frequency-weighted mean dominated by common words; the other is an unweighted coverage rate over the entire vocabulary. We will define each precisely, build a concrete case where they disagree, explain why the disagreement is the interesting signal, and connect it to what teams actually do in production.
Two different statistics of one tokenization
Fertility is the average number of tokens per word, measured over a representative corpus. Because it is a corpus average, it is implicitly weighted by how often each word appears. The most frequent words contribute the most to the number.
STRR, the single-token retention rate, is the fraction of a language's distinct vocabulary that the tokenizer keeps as a single token. It is computed over the word list, not the corpus, so every word counts once regardless of frequency.
The contrast is the heart of the matter. Fertility is a weighted mean; STRR is an unweighted coverage rate. They can move in opposite directions, and when they do, the gap between them is exactly the information you were missing from either one alone.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Tokenizer fairness audits of OpenAI's o200k_base report fertility and retention together per language to show whether multilingual gains reach the rare-word tail, not just common words.
- The BLOOM and Aya multilingual projects evaluated coverage breadth alongside average fertility when designing balanced tokenizers for low-resource languages.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you compute STRR in practice for a language whose word list you do not cleanly have?
Derive a distinct-word list from a representative corpus with a defined segmentation convention, then count one-token words over distinct words, not over token occurrences.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating STRR as just another name for low fertility, when one is a frequency-weighted average cost and the other is unweighted coverage breadth across the whole vocabulary.
60 second bullets to scan on the way to the call.
STRR as fraction of vocabulary words that are single tokens
Fertility as frequency-weighted average tokens per word
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.