A product manager claims 'Our 128k context model can hold a 128,000-word document.' You are building a multilingual RAG pipeline serving English, French, Japanese, and Arabic users. Correct the PM's claim for English and explain how token budgeting must differ for each language. Then outline a practical method for estimating per-language token budgets.
Free · 2 AI evals / day
First, correct the framing. A context window is counted in **tokens**, the small text pieces a tokenizer produces, not in words. In English, one word is about 1.3 tokens on average, so a 128k-token window holds roughly 96,000 to 98,000 English words, not 128,000. The PM is over by about a third.
It gets worse in other languages. The tokens per word ratio is called fertility, and it is not constant. French sits a little above English. Japanese and Arabic can be three to five times higher, because their characters take more bytes and the tokenizer learned fewer dedicated pieces for them. The same 128k window might hold only 25,000 to 40,000 Japanese or Arabic words.
So to build a budget estimator, do not guess from word counts. Measure. Take real documents in each language, run them through the actual tokenizer, and record the average tokens per word per language. Store those numbers and use them to compute the true budget before every call.
Start by fixing the premise: context windows are measured in **tokens**, not words. English averages about 1.3 tokens per word, so 128k tokens holds roughly 96,000 to 98,000 English words. The PM's claim is off by about a third even before we leave English.
The variable that breaks the budget across languages is fertility, the tokens per word ratio. French runs slightly higher than English from accents and morphology. Japanese and Arabic can run three to five times higher, because their characters consume more bytes and the English-biased vocabulary has fewer merges for them. So 128k tokens that holds 96k English words might hold only 25k to 40k Japanese or Arabic words.
The practical estimator is empirical, not rule of thumb. Sample about 1,000 real documents per language from your actual distribution. Tokenize each with the model's canonical tokenizer, `tiktoken` for OpenAI models. Compute mean tokens per word per language and store a budget table. Then, at request time, count tokens directly before each API call rather than estimating from characters or words.
The reason to measure rather than assume is that fertility depends on your specific domain vocabulary, not just the language.
Quick reference
TL;DR
Context windows are measured in tokens, not words, and fertility varies 1.3x to 5x by language, so budget by measuring tokens per word per language with the real tokenizer, never from word counts.
Imagine a suitcase that holds a fixed number of folded shirts, not a fixed number of outfits. English outfits fold small, so many fit. Japanese and Arabic outfits fold bulkier, so fewer fit even though the suitcase is the same size. The suitcase is the context window, the shirts are tokens, and the outfits are words. The product manager assumed one word equals one shirt, but it does not: a word can take one shirt in English and three to five in Japanese. To pack right, you weigh real clothes per language instead of guessing.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
An English-only product with stable, short inputs A single measured ratio is enough; a full per-language table is over-engineering there.
Budgeting before you have any representative data Empirical fertility depends on your corpus, so a table built on borrowed numbers can be badly off.
Reasoning across a tokenizer change without re-measuring Fertility shifts with the encoding, so an old table cannot just be scaled to a new model.
Quick scan · 2–4 min
Everything important, quickly.
3 min: tokens not words + English ~1.3x correction + fertility varies by language + Japanese/Arabic 3-5x + sample real docs + per-language table + p95 buffer + rate limits.
Code example
python
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o") # o200k_base
def fertility(samples: list[str]) -> float:
total_tokens = sum(len(enc.encode(doc)) for doc in samples)
total_words = sum(len(doc.split()) for doc in samples)
return total_tokens / max(total_words, 1)
# Build a per-language table from real sampled documents
budget_table = {
lang: fertility(docs) for lang, docs in samples_by_language.items()
}
# At request time, count tokens directly (never estimate from words)
def fits(doc: str, window: int = 128_000) -> bool:
return len(enc.encode(doc)) <= window
Side-by-side
Language
Approx. fertility (tokens/word)
Words in a 128k window
English
~1.3x
~96,000
French
~1.1-1.5x
~85,000-115,000
Japanese
~3-5x
~25,000-40,000
Arabic
~3-5x
~25,000-40,000
Real-world examples
Real products, models, and research that use this idea.
1OpenAI bills GPT-5.5 by token, so a multilingual RAG product must size retrieval by tokens per language.
2Perplexity retrieves and stuffs documents into the context window, where high-fertility languages eat the budget faster.
3tiktoken is the canonical way to measure per-language fertility for OpenAI models before sizing a budget.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you handle documents that mix languages within a single chunk?
Sign in or go Pro to see the approach for this follow-up.
QHow does switching from cl100k_base to o200k_base change your budget table?
Sign in or go Pro to see the approach for this follow-up.
QHow would you prevent a high-fertility language from exhausting a tokens per minute rate limit?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Telling the PM the model holds 128,000 words. Context windows count tokens, and English alone runs about 1.3 tokens per word, so the real figure is closer to 96,000 words.