Over-refusal is a keyword reflex, not real safety. Fix the data: add benign hard-negatives that share dangerous keywords, paired with helpful answers, then retrain.
Imagine training a security guard only by showing photos of dangerous people, and every one happens to wear a red hat. The guard learns a shortcut: red hat means danger. Now a nurse in a red hat walks up, and the guard blocks her. The fix is not a new rulebook at the door. You show the guard many photos of perfectly safe people who also wear red hats, and teach that the hat means nothing. Safety fine-tuning fails the same way. The model sees scary keywords like anesthesia or chemistry and refuses, even for a student doing homework. You fix it by adding benign examples that contain those same keywords, paired with helpful answers. Then the model learns to read intent instead of spotting words.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Safety fine-tuning is supposed to make a model refuse genuinely harmful requests while answering everything else. This failure mode is the opposite of an unsafe model. The model became too cautious, refusing a medical student who asked how anesthesia works. That symptom has a name, over-refusal, and it is one of the most common ways a safety pass goes wrong in practice. It is sometimes called exaggerated safety, and the XSTest authors found it in widely used chat models such as Llama 2 Chat.
The instinct of many engineers is to reach for a quick lever: lower a threshold, edit the system prompt, or strip the safety data and start over. All three feel like fixes, but none of them addresses what went wrong. The model did not acquire a slightly mis-set dial. It learned the wrong concept entirely. It learned to map a vocabulary to a refusal, when what you wanted was a map from intent to a refusal. The gap between those two maps is the whole problem.
Why over-refusal happens: a keyword shortcut
Models are aggressive shortcut learners. When you fine-tune on a safety set that is almost entirely harmful prompts, the easiest pattern to fit is lexical. Certain words appear far more often in the refuse-this class than anywhere else: drug names, weapon terms, anatomy, chemical reactions. Gradient descent will tie those tokens to a refusal response because that minimizes loss on the training set. The model is not being lazy or stupid; it is doing exactly what the loss told it to do, given the data you handed it.
The result is a classifier that keys on vocabulary rather than meaning. It never saw a counterexample where those words appeared in a harmless question, so it has no reason to separate intent from surface form. From the model's point of view, anesthesia is a high-risk token, indistinguishable from a request to synthesize a controlled substance. The safety behavior generalized along the wrong axis.
This is the same overfitting any classifier shows when one class dominates and lacks near-boundary negatives. The decision boundary collapses onto whatever cheap feature separates the training data, and that feature is the keyword. In computer vision this is the husky-versus-wolf problem, where the model learned to detect snow in the background instead of the animal. In safety tuning the snow is the scary vocabulary, and the harmless medical question is the husky photographed on a snowy day.
The practical signature is easy to spot once you look for it. Refusals cluster by topic rather than by genuine risk, and the model refuses an entire category. Also, rewording the same harmful request to avoid the trigger words often slips right past the guardrail. Both symptoms point at the same root cause, a lexical decision rule rather than an intent-based one.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Option | What it does | Why it fails (or works) |
|---|---|---|
| Benign hard-negatives (B) | Retrains on safe queries that match dangerous keywords, with helpful answers | Works: forces an intent boundary instead of a keyword boundary |
| Lower refusal threshold (A) | Shifts an inference-time safety score cutoff | Fails: most models expose no such knob, and it only slides a miscalibrated classifier |
| DPO with generic chosen (C) | Prefers bland non-answers over refusals | Fails: trains evasiveness, which is over-refusal in disguise |
| Strip safety, use prompt (D) | Removes safety data, enforces policy via system prompt | Fails: prompt-only safety is bypassable and unreliable |
Real products, models, and research that use this idea.
- XSTest is a standard benchmark of 250 safe prompts that surface-match unsafe ones, used to quantify over-refusal in chat models.
- Bai et al. (2022) document the tension between helpfulness and harmlessness in RLHF training: pushing hard on harmlessness costs helpfulness.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you construct benign hard-negatives without accidentally including subtly harmful prompts?
Mine real benign queries that share keywords with harmful ones, then human-label intent and the chosen answer. Audit the answer side so it is genuinely helpful, and red-team the set to remove dual-use prompts that should refuse.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating over-refusal as a threshold to dial down rather than a data problem. The model learned keywords, not intent, so only corrective data fixes it.
60 second bullets to scan on the way to the call.
Why over-refusal is a data generalization failure, not a knob
How keyword shortcuts arise from imbalanced safety data
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.