Sarvam AI

Select the correct comparisons of an MoE transformer versus a dense transformer at equal active parameters

You hear 'fertility is 2.5 for Hindi on cl100k_base'. What does that number mean and is it good?

You want to add 5,000 Hindi medical terms to Llama 3 8B. Walk through the procedure and the risks.

You are building a 7B model for 25 languages including Hindi, Arabic, and Swahili. Design a fair tokenizer and name what it costs.

The same sentence in English, Spanish, Hindi, and Japanese. Predict the token ratio on o200k_base for each.

What is the 'token tax' and how does it create downstream accuracy disparities across languages?

What primarily causes the 'token tax' for low-resource languages in BPE tokenizers?

What is STRR and how does it reveal poor tokenizer coverage that fertility alone misses?

What is STRR and how does it complement fertility as a tokenizer fairness metric?

Define 'fertility' as a tokenizer metric and explain why high fertility for a language is both a fairness and a model performance problem.