A product manager claims 'Our 128k context model can hold a 128,000 word document.' You are building a multilingual RAG pipeline serving English, French, Japanese, and Arabic users. Correct the PM's claim for English and explain how token budgeting must differ for each language. Then outline a practical method for estimating per language token budgets.
Context windows are measured in tokens, not words, and fertility varies 1.3x to 5x by language, so budget by measuring tokens per word per language with the real tokenizer, never from word counts.
Imagine a suitcase that holds a fixed number of folded shirts, not a fixed number of outfits. English outfits fold small, so many fit. Japanese and Arabic outfits fold bulkier, so fewer fit even though the suitcase is the same size. The suitcase is the context window, the shirts are tokens, and the outfits are words. The product manager assumed one word equals one shirt, but it does not: a word can take one shirt in English and three to five in Japanese. To pack right, you weigh real clothes per language instead of guessing.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Every multilingual LLM product eventually meets this conversation. A stakeholder reads '128k context' and pictures 128,000 words of document. The number is real, but the unit is wrong, and the gap between tokens and words is where budgets blow up.
The interview version of this question is checking two things. Can you correct the unit error cleanly, and do you understand that the correction is not a single number but a per-language one? Anyone can say 'tokens, not words.' Fewer people can explain why Japanese fits a third of what English does in the same window, and fewer still can describe a measurement process that survives contact with real data.
We will correct the English claim, define fertility and show why it swings so widely across scripts, then build a concrete, measured estimator and cover the production details that keep a multilingual RAG pipeline from silently overflowing its window or its rate limit.
The stakes here are not academic. A retrieval pipeline that assumes word-based budgeting will, for a high-fertility language, either truncate retrieved context and degrade answer quality, or overflow the window and trigger an API error mid-request. Both failure modes show up only in production and only for the languages you were least likely to test thoroughly. Getting the unit and the per-language factor right at design time is what prevents a class of bugs that are painful to diagnose after launch.
Why tokens and words are not interchangeable
A context window is a hard limit on the number of tokens the model processes in one request. Tokens are the subword pieces the tokenizer emits, and they do not line up one to one with words.
In English the average is about 1.3 tokens per word. Common words are a single token, but plurals, suffixes, rare words, and punctuation push the average above one. So a 128k-token window holds roughly 96,000 to 98,000 English words, not 128,000.
That is already a 33 percent correction, and it is the easy part. The PM's mental model has the right number attached to the wrong unit. Fixing the unit is step one; the harder insight is that the conversion factor itself is not a constant.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o") # o200k_base
def fertility(samples: list[str]) -> float:
total_tokens = sum(len(enc.encode(doc)) for doc in samples)
total_words = sum(len(doc.split()) for doc in samples)
return total_tokens / max(total_words, 1)
# Build a per-language table from real sampled documents
budget_table = {
lang: fertility(docs) for lang, docs in samples_by_language.items()
}
# At request time, count tokens directly (never estimate from words)
def fits(doc: str, window: int = 128_000) -> bool:
return len(enc.encode(doc)) <= window| Language | Approx. fertility (tokens/word) | Words in a 128k window |
|---|---|---|
| English | ~1.3x | ~96,000 |
| French | ~1.1-1.5x | ~85,000-115,000 |
| Japanese | ~3-5x | ~25,000-40,000 |
| Arabic | ~3-5x | ~25,000-40,000 |
Real products, models, and research that use this idea.
- OpenAI bills GPT-5.5 by token, so a multilingual RAG product must size retrieval by tokens per language.
- Perplexity retrieves and stuffs documents into the context window, where high-fertility languages eat the budget faster.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you handle documents that mix languages within a single chunk?
Tokenize the mixed chunk directly rather than blending per-language averages, since fertility is non-linear across script boundaries.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Telling the PM the model holds 128,000 words. Context windows count tokens, and English alone runs about 1.3 tokens per word, so the real figure is closer to 96,000 words.
60 second bullets to scan on the way to the call.
Why a context window counts tokens, not words
Approximate English tokens per word and the corrected word capacity
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.