BPE grows the vocabulary by adding merges. Unigram does the opposite. Explain the difference.

**BPE and Unigram LM** are two ways to train a subword tokenizer, and they differ in almost every meaningful dimension despite both producing a finite vocabulary of subword pieces. BPE is additive and greedy. The training loop starts from the base alphabet (bytes or characters), counts every adjacent pair across the corpus, picks the most frequent pair, merges it into a new piece, and repeats until the vocabulary reaches the target size. At encoding time, BPE applies the learned merges greedily and produces exactly one segmentation per input string. Unigram LM is subtractive and probabilistic. The training loop starts from a large candidate vocabulary that includes many candidate substrings. It then iteratively prunes tokens whose removal would hurt the corpus likelihood least, where likelihood is measured under a unigram language model assumption. Training uses the EM algorithm to refine token probabilities. At encoding time, Unigram LM picks the most probable segmentation by default but can also sample alternative segmentations from the distribution of valid splits. In 2026, BPE is the dominant generative-LLM choice (GPT-5.5, Llama 3 and 4, Mistral, DeepSeek). Unigram LM lives inside SentencePiece's default mode and was the algorithm behind Llama 2, Gemma, and T5-family models.

**BPE and Unigram LM** are two subword tokenizer training algorithms that differ along three axes: direction, determinism, and output flexibility. BPE is additive and greedy. Starting from the base alphabet, it repeatedly picks the most frequent adjacent pair, merges it everywhere, and adds it as a new vocabulary entry. The training artifact is an ordered merge list. At encoding time, BPE applies the merges greedily and produces a single deterministic segmentation per input string. Unigram LM is subtractive and probabilistic. Starting from a large candidate vocabulary (typically built by enumerating frequent substrings), it iteratively prunes tokens to maximize corpus log-likelihood under a unigram model. Training is run via the EM algorithm: the E step computes expected counts of each token given current probabilities, the M step updates probabilities to match those counts, and pruning removes the lowest-utility tokens. The training artifact is a vocabulary with associated token probabilities. At encoding time, Unigram LM picks the maximum-likelihood segmentation by default but supports subword regularization, which samples alternative segmentations from the distribution of valid splits. Sampling at training time gives the downstream model multiple views of the same input string, a free data-augmentation strategy introduced by Kudo in 2018. In 2026, BPE dominates generative LLMs: GPT-5.5 via tiktoken o200k_base, Llama 3 and 4 with byte-level BPE, Mistral and DeepSeek using BPE-family encoders. Unigram LM is the default in Google's SentencePiece library and was the algorithm behind Llama 2, Gemma, and the T5 family. The choice between BPE and Unigram LM is one of the actual live tokenizer-design decisions in 2026, unlike the mostly-historical BPE-versus-WordPiece split.