Code-heavy tokenizer corpora can improve syntax packing but hurt what elsewhere?

Tokenizer vocabulary is a finite budget. If you allocate more merges to code patterns, code strings become shorter in tokens, which is great for code tasks. But that same allocation can reduce how well the tokenizer captures rare natural-language fragments, multilingual morphology, or domain-specific non-code terms. Those segments may split into more pieces, increasing sequence length and making learning less efficient. So the hurt is outside code: non-code token efficiency and downstream quality can degrade if code-heavy allocation is too aggressive. Strong teams validate tokenizer changes by domain, not only by average compression.

The missing piece is **vocabulary opportunity cost**. A code-heavy tokenizer corpus shifts merge priorities toward operators, delimiters, and identifier fragments. That lowers code fragmentation and improves context packing on repositories. However, tokenizer capacity is fixed, so those extra code-friendly merges displace potential merges for non-code patterns. The usual casualty is long-tail natural language efficiency: rare morphemes, multilingual variants, and domain terms get split more often. That raises token counts, increases effective training cost for those domains, and can reduce quality there even if code metrics improve. Interview-grade answer: code-heavy tokenizer tuning helps code compression but can hurt non-code representation efficiency unless you monitor per-domain tokenization diagnostics.