A byte level BPE tokenizer (like GPT-2's) has never seen any emoji during training, so no emoji merge rules were learned. The rainbow flag emoji 🏳️🌈 is a ZWJ sequence: it encodes as the UTF-8 bytes for 🏳 (white flag, 4 bytes) + ️ (variation selector, 3 bytes) + ZWJ U+200D (3 bytes) + 🌈 (rainbow, 4 bytes) = approximately 14 bytes total. Predict how many tokens this emoji produces.
With no learned merges, byte-level BPE falls back to one token per byte, so the ~14-byte rainbow flag ZWJ sequence becomes roughly 14 tokens.
Imagine a typist who knows shortcut keys for common words. Type 'hello' and one key produces the whole word. But hand them a strange symbol they have never practiced, and they have no shortcut, so they spell it out one keystroke at a time. The rainbow flag emoji is that strange symbol. To the computer it is really four little pictures glued together with an invisible 'joiner', and all of it adds up to about 14 tiny byte-pieces. With no learned shortcut, the tokenizer types each byte-piece on its own, so one emoji costs about 14 keystrokes, while the word 'hello' costs barely one or two.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
This question looks like arithmetic, but it is really testing whether you separate three things people routinely conflate: how an emoji is drawn, how it is encoded, and how it is tokenized. Get those three layers straight and the count falls out.
The rainbow flag 🏳️🌈 is the perfect trap because it renders as one tidy glyph. Intuition screams 'one token'. The reality is that it is a small program of four Unicode codepoints stitched together with an invisible joiner, and a tokenizer that never learned it will spell it out byte by byte.
We will work down the layers: the merge rule that decides everything, the Unicode structure of the emoji, the UTF-8 byte count, the production lesson about why emoji-heavy text quietly blows up token budgets, and finally why the answer is deliberately a range rather than a single integer.
The deeper habit this question builds is keeping three layers distinct that beginners blur together: rendering (how many cells the glyph fills), encoding (how many bytes UTF-8 uses), and tokenization (how many token IDs the model sees). Almost every wrong answer here comes from collapsing those three into one. Keep them separate and the count is mechanical.
The rule that decides the count
Byte-level BPE has exactly two ingredients: a base vocabulary of all 256 byte values, and a learned list of merges that combine frequent adjacent pairs into single tokens.
The merges are where compression comes from. 'hello' is 5 bytes, but because the sequence appeared constantly in training, merges collapse it to one or two tokens.
Now flip it around. If the tokenizer never saw a pattern, it learned no merge for it, and there is nothing to combine. The tokenizer falls back to its base vocabulary and emits one token per byte. That gives the floor for any unseen input.
The whole problem now reduces to one question: how many UTF-8 bytes is this emoji?
It helps to see this as the same mechanism behind the non-Latin token tax, just at its most extreme. Arabic fragments because few Arabic merges were learned; an unseen emoji fragments completely because zero merges were learned. The emoji is the limiting case where the merge table offers no help at all, so the token count equals the byte count exactly. Once you see byte fallback as a spectrum from 'fully merged English' to 'fully fragmented unseen symbol', the emoji answer is just the far end of that spectrum.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
emoji = "🏳️🌈" # rainbow flag, a ZWJ sequence
# Inspect the Unicode structure: 4 codepoints joined by ZWJ.
print([hex(ord(c)) for c in emoji])
# ['0x1f3f3', '0xfe0f', '0x200d', '0x1f308']
# flag VS-16 ZWJ rainbow
print(len(emoji.encode("utf-8"))) # ~14 bytes
# With no learned merges, a byte-level BPE tokenizer
# emits ~1 token per byte -> ~14 tokens.Real products, models, and research that use this idea.
- Chat products on GPT-5.5 see emoji-dense messages inflate token counts, since each unseen ZWJ emoji can cost a dozen tokens of input.
- tiktoken and HuggingFace tokenizers both expose byte fallback, so any emoji always encodes but rare ones expand to many byte tokens.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does a plain 🌈 sometimes cost fewer tokens than the rainbow flag 🏳️🌈 in a modern tokenizer?
Compare merge coverage: common single-codepoint emoji often earned merges, while rare multi-codepoint ZWJ sequences still fall back to bytes.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Counting the emoji as 1 token because it looks like one glyph, when it is a 4-codepoint ZWJ sequence of ~14 UTF-8 bytes with no merges to compress it.
60 second bullets to scan on the way to the call.
Why no merges means a floor of one token per byte
The four codepoints inside the rainbow flag ZWJ sequence
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.