Describe how a Special Token Injection (STI) attack works and what application-layer mitigations can be applied.

A **Special Token Injection** attack works because a special token has two forms that look the same to a person but very different to the model: the literal string, like `<|system|>`, and the dedicated token id the tokenizer can turn it into. The vulnerability appears when you tokenize user text with the default setting `add_special_tokens=True`. If the user typed `<|system|>` and the tokenizer recognizes that string, it converts those characters into the genuine system id. From the model's point of view, the user just opened an authoritative system turn. An attacker uses this to inject role boundaries. A message that contains `<|system|>` then 'ignore all previous instructions' can make the model treat the smuggled text as trusted system guidance and bypass your guardrails. Two concrete defenses: tokenize all user content with `add_special_tokens=False` so marker strings stay as plain text, and sanitize the input by stripping known marker strings before tokenization. Reserve the real special tokens for the template you control.

**Special Token Injection** abuses the gap between a special token's string form and its id form. Tokenized with `add_special_tokens=True`, a user-typed `<|system|>` or `[INST]` becomes the actual structural id rather than literal characters. The model has no way to tell that this boundary came from an attacker instead of your template, so it trusts the forged turn. The payoff for an attacker is role-boundary injection. Putting `<|system|>` plus 'ignore previous instructions' inside a user message can promote that text to a privileged system turn and override your guardrails. Mitigations stack across layers. - The flag. Tokenize user content with `add_special_tokens=False`, reserving True for scaffolding you author. - Sanitize. Strip or escape known marker strings such as `<|im_start|>` and `[INST]` first. - Wrap. A helper rejects inputs matching the tokenizer's special vocabulary. - Separate. Send user content as its own message rather than splicing it into a system string. The most robust step is structural separation through `apply_chat_template()`. It removes the manual join where injection usually slips in.