**BPE and WordPiece** are both subword tokenizer training algorithms. They both start from a base alphabet (bytes or characters), and they both grow a vocabulary by repeatedly picking two adjacent pieces and merging them into a new piece. They differ in one place: how they pick which pair to merge next.
BPE picks the pair that appears most often in the training corpus. Count every adjacent pair across the corpus, pick the pair with the highest count, merge it, repeat. That is the whole algorithm.
WordPiece picks the pair whose merge would most increase the likelihood of the corpus under a unigram language model. In practice this works out to maximizing the score freq(AB) divided by freq(A) times freq(B). A pair that occurs often, but where each piece is rare on its own, scores high. A pair where both pieces are already very common scores low, even if the pair itself is frequent.
In 2026, BPE is what generative LLMs use: GPT-5.5 (tiktoken cl100k_base and o200k_base), Llama 3 and 4, Mistral. WordPiece is the BERT-family choice and shows up mostly in research and legacy encoder pipelines.
**BPE and WordPiece** are both iterative subword tokenizer training algorithms. They share the same skeleton: start from a base alphabet, count adjacent pairs across the corpus, pick a winning pair, merge it everywhere, repeat until the vocabulary reaches the target size. The headline difference is the scoring function used to pick the winning pair.
BPE picks the pair with the highest raw frequency. This is a pure count-based criterion, originally borrowed from data compression (Gage 1994, Sennrich et al. 2016). The selected pair is whichever adjacency shows up most often in the current corpus state.
WordPiece picks the pair whose merge most increases corpus likelihood under a unigram language model. The full derivation reduces to picking the pair (A, B) that maximizes freq(AB) divided by freq(A) times freq(B). This ratio is high when AB occurs together far more often than chance would predict given how often A and B occur separately. WordPiece therefore prefers merges that are statistically surprising, not merely common.
In 2026 production, BPE dominates generative LLMs: GPT-5.5 uses tiktoken with the o200k_base BPE encoder, Llama 3 and 4 use a 128K BPE vocabulary, Mistral uses a SentencePiece BPE. WordPiece is the BERT-family algorithm and is now mostly research or legacy encoder tooling. Both apply their learned merges greedily at encoding time; the algorithmic split is purely about training.
Quick reference
TL;DR
BPE picks the next merge by raw pair frequency; WordPiece picks the merge that maximally increases corpus likelihood, scored as freq(AB) over freq(A) times freq(B).
Picture a kid stacking blocks. Both kids start with a pile of letter blocks and a rule: each turn, glue two blocks together to make a bigger block, and add the new shape to your toolbox. The first kid (BPE) glues whichever two blocks happen to sit next to each other most often. Simple counting. The second kid (WordPiece) is fussier. It only glues two blocks together if the pair shows up much more often than you would expect from how often each block shows up on its own. That ratio rewards merges that are genuinely surprising. After many turns, both kids have a toolbox of bigger blocks they can use to build any word. They tend to pick similar blocks for common words but diverge on rarer combinations, which is why a text tokenized with BPE and the same text tokenized with WordPiece end up looking subtly different.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Choosing a tokenizer for a brand new generative LLM in 2026. Neither pure character-level BPE nor classical WordPiece is the right call; byte-level BPE has won this segment because it eliminates the unknown-token problem and matches the GPT and Llama defaults developers already expect.
Picking a scoring function based on which algorithm gives shorter token sequences. Sequence length is dominated by vocabulary size and corpus match, not by the merge criterion; you cannot reliably predict that BPE will be shorter or longer than WordPiece on a given corpus.
Using WordPiece for a sampling or data augmentation pipeline that wants multiple segmentations. WordPiece, like BPE, produces a single deterministic segmentation per input; Unigram LM is the tokenizer that supports subword regularization with multiple sampled segmentations.
Quick scan · 2–4 min
Everything important, quickly.
4 min: state the shared training skeleton, name BPE's frequency criterion, name WordPiece's likelihood criterion with the freq(AB) over freq(A) times freq(B) ratio, explain why the ratio rewards surprising merges, and place both algorithms in the 2026 production landscape with one or two model examples each.
Real-world examples
Real products, models, and research that use this idea.
1GPT-5.5 and o3 use BPE via tiktoken's o200k_base encoding with a 200K vocabulary; no WordPiece anywhere in the OpenAI stack in 2026.
2Llama 3 and Llama 4 use BPE with a 128K vocabulary, having switched away from the SentencePiece tokenizer used in Llama 2.
3BERT, DistilBERT, and the legacy encoder family still ship with WordPiece tokenizers; this is the main place WordPiece appears in 2026 production code.
4The HuggingFace tokenizers library exposes both BPE and WordPiece as first-class training algorithms, which is how the two stay accessible side by side.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QWhere does the freq(AB) over freq(A) times freq(B) formula come from?
Sign in or go Pro to see the approach for this follow-up.
QWhy have most generative LLMs converged on BPE rather than WordPiece?
Sign in or go Pro to see the approach for this follow-up.
QIf both produce one segmentation per input, where does any meaningful difference between encoded outputs come from?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Saying BPE and WordPiece differ in how they segment at inference time. They differ in how they pick merges during training; both apply learned merges greedily at encoding time.