Spot the failure mode: a tokenizer trained with NFC normalization is deployed with an NFKC preprocessing pipeline.

The flawed claim is that token IDs will be equivalent and behavior unaffected because both are valid normalizations. That is wrong. Unicode normalization rewrites text into a canonical form before tokenizing. NFC does the lighter cleanup, handling only canonical equivalences like the two ways to write an accented letter. **NFKC** does that plus compatibility folding: it rewrites ligatures like `fi` into `fi`, half-width katakana into full-width, circled numbers into plain digits, and superscripts into base digits. Because NFKC changes the actual characters for those cases, it produces different bytes than NFC. The tokenizer here was trained on NFC text, so it learned merge rules for NFC byte patterns. Feeding it NFKC text at inference means those compatibility characters arrive in a form the model rarely trained on. Nothing crashes. The model just behaves worse on text containing those characters, which is exactly the silent failure mode the statement misses.

The error is the conclusion that NFC and NFKC produce equivalent token IDs. They do not, and the consequence is a **silent normalization mismatch**. NFC applies canonical decomposition then composition, which only unifies characters Unicode defines as truly equal. NFKC additionally applies compatibility decomposition. That extra step rewrites ligatures, half-width and full-width forms, circled digits, and superscripts into base forms. For any input containing those characters, NFKC yields a different byte sequence than NFC. The tokenizer was trained on NFC text, so its merges and embeddings are tuned to NFC byte patterns. Serving NFKC-normalized input means the model receives byte sequences it rarely saw for compatibility characters. Tokenization shifts, and the resulting embeddings are undertrained. The defining property is that it fails silently. There is no exception and no `[UNK]` flood, just degraded behavior concentrated on Japanese, mathematical notation, and formatted text where compatibility characters cluster. The fix is to pin one normalization form across training and serving, and assert it at the tokenizer boundary so a library default change fails loudly.