Same topic, related formats. Practice these next.
Fine-tuning on docs teaches the confident style of expert answers, not the facts themselves. The model gets surer of itself while staying ignorant, so it confabulates fluently.
Imagine a student who memorises how confident, polished medical answers sound but never studies the textbook. After a weekend of copying how doctors phrase things, they sound exactly like a doctor: calm, certain, full of jargon. But they still don't know the facts. So when you ask a real question, they answer in that same confident doctor voice and just make things up. Fine-tuning on a few thousand company documents does the same thing. It teaches the model the shape and tone of expert answers, not the underlying knowledge. The model becomes more sure of itself without becoming more correct. Confidence goes up, knowledge stays flat, and confident wrong answers are exactly what hallucination is.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: what SFT teaches well vs poorly + the pretraining-scale asymmetry + confidence inflation from the loss + catastrophic forgetting + why each distractor fails + the retrieval plus abstention fix.
| What you want | Fine-tuning delivers | Better tool |
|---|---|---|
| Confident domain tone and format | Yes, efficiently from a few thousand examples | Fine-tuning |
| Memorising specific facts in the docs | Poorly; needs pretraining-scale repetition | RAG |
| Freshness when docs change | No; weights are stale at training time | RAG |
| Saying 'I don't know' on unknown queries | Only if you train abstention explicitly | Abstention fine-tune plus RAG |
Real products, models, and research that use this idea.
What an interviewer would ask next. Try answering before peeking at the approach.
Red flags and common mistakes that signal junior thinking. Click to expand.
Assuming fine-tuning on documents injects their facts into the model. It mostly injects the style of confident answers, which raises confabulation when the underlying knowledge never actually transferred.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.