A fresh mix across formats every day. Pro weaves your weak topics into each set.
01 / 10
0 completed
What does '20 tokens per parameter' mean in Chinchilla style planning?
flashcard
Answer revealed
Training compute is about 6 x parameters x tokens FLOPs. For a fixed budget, the Chinchilla fits put the lowest training loss at roughly 20 training tokens per parameter: a 70B model on about 1.4T tokens. It is a compute-optimal baseline for training cost only. It leaves out serving cost and data limits, so many models train far past it.