Why is it not possible to simply swap a pretrained LLM's tokenizer for one with a larger vocabulary?

A tokenizer turns text into a list of integer **token IDs**, and the model learns one embedding vector for each ID during pretraining. The input embedding table has shape `[vocab_size, d_model]`, which means one row per token ID. The problem with swapping the tokenizer is that the IDs are not just labels. They are the addresses the model uses to look up learned meaning. If a new tokenizer assigns the word "hello" to a different ID, the model still fetches the embedding row at the old address, which now belongs to a different token entirely. There is a second component with the same dependency. The output head, called `lm_head`, has shape `[d_model, vocab_size]` and turns the model's final hidden state into a score for every possible next token. A bigger vocabulary means new rows here too, and those new rows have no trained weights. So it is not a memory limit or a licensing rule. The learned weights are tied to a specific ID layout, and changing that layout breaks the lookup on both ends of the model. That is why the tokenizer is effectively chosen once, at the start of pretraining, and frozen after.

You cannot hot-swap a tokenizer because two model components are indexed by token ID, and changing the vocabulary scrambles that index. The input embedding table has shape `[vocab_size, d_model]`, one learned row per ID. The output projection `lm_head` has the mirror shape `[d_model, vocab_size]`. Both were trained against one specific mapping from token to ID. When a new tokenizer remaps IDs, the row that used to mean "the" now sits at a different position. So every embedding lookup returns the wrong learned vector. A larger vocabulary also adds brand-new IDs that have no trained rows at all, on both the input and output side. The key insight is that token IDs are not arbitrary labels you can relabel. They are the literal addresses the model uses to fetch and produce meaning, and the addresses are baked into the weights. So the practical answer is that tokenizer choice is **sticky**: it is fixed at the start of pretraining. To change it you either retrain from scratch, or run a vocabulary-adaptation step that initializes the new rows and continues training, which is far cheaper but still not free.