Give three concrete situations where fine-tuning is the wrong tool. For each, explain why FT specifically fails and what to reach for instead.
Don't fine-tune when prompting already works, when knowledge changes often, when you have under 500 examples, or when there is no eval to prove it helped.
Fine-tuning is like sending a chef to a months-long cooking school to permanently change how they cook. That makes sense if you want a new style of food forever. But it is the wrong move for some problems. If today's menu changes daily, you do not retrain the chef each morning: you just hand them today's menu, which is retrieval. If the chef already cooks the dish well from a written recipe, retraining is wasted money and might make their other dishes worse. And if you only have three example plates to learn from, the school has nothing solid to teach from. So before retraining, ask: is this really a cooking-habit problem, or just a missing-menu problem?
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Fine-tuning has a gravitational pull in interviews and in real teams. It feels like the serious, grown-up fix: the model is failing, so we will retrain it. That instinct is wrong more often than it is right, and recognising the wrong cases is exactly the judgment a senior interviewer is probing. The question is not whether you can fine-tune. It is whether you can resist fine-tuning when something cheaper and more reliable would do the job.
The core move is to stop treating fine-tuning as a generic upgrade and start diagnosing the problem type. Fine-tuning edits the model's parameters, which makes it the right lever for one specific thing: changing behavior and style that generalise across many inputs. Brand voice, refusal policy, a consistent output register, domain reasoning patterns that need new internal abstractions. Those are real fine-tune jobs. The moment the real problem is fresh facts, a thin last-mile gap, too little data, or a missing measurement, fine-tuning becomes the wrong tool, sometimes an actively harmful one.
There is also an asymmetry worth internalising. A fine-tune is expensive to run, slow to iterate, hard to roll back, and capable of silently degrading capabilities you were not even testing. The alternatives, retrieval, prompt iteration, structured-output mode, and in-context learning, are cheap, fast, reversible, and observable. When two tools could plausibly solve a problem, the reversible one wins by default. You escalate to fine-tuning only after the cheap tools have demonstrably plateaued.
This deep dive walks the three canonical anti-patterns the question asks for, adds the two gates that quietly disqualify most fine-tune proposals, and ends with a worked decision so you can run the diagnosis live in an interview.
Case 1: frequently-updated knowledge belongs in retrieval
Fine-tuning bakes facts into the weights at training time. The artifact you ship is a snapshot of what was true when the run finished. That is fine for behavior, which is stable, and ruinous for knowledge, which moves. A model trained to know your Q2 pricing is wrong the instant pricing changes, and nothing short of another training run will fix it.
If your knowledge base updates weekly, daily, or hourly, a fine-tuned model is stale almost immediately. The only way to keep weights current is to retrain on every change, which is operationally absurd at any real cadence. There is a subtler failure too. Fine-tuning on facts the base model never saw can teach it to confabulate confidently, because it learns the surface shape of an answer without a reliable internal source for the specifics. You get fluent, well-formatted, wrong answers, which are the most dangerous kind.
The right tool is retrieval. Keep the model frozen, index the corpus, and retrieve the relevant chunks at query time. Updates to the index land in seconds, citations fall out naturally, and there is a clean audit trail of exactly what context produced what answer. Compliance regimes that forbid training on customer data also push you here, since retrieval keeps records in a controlled, auditable store rather than smeared across the weights where membership-inference attacks could extract them.
The diagnostic question is simple: if updating the answer would require retraining, the knowledge is in the wrong place. Facts go in the index, not the weights.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Symptom | Why fine-tuning fails | Reach for instead |
|---|---|---|
| Knowledge updates weekly or faster | Facts freeze in weights at train time and go stale | RAG with a live index, retrieved at query time |
| Few-shot prompt already above 85 percent | Low leverage plus regression risk for a small gain | Prompt iteration, better examples, structured output |
| Under 500 clean training pairs | Run under-converges, often below base plus prompt | In-context learning, or gather and curate more data |
| Need tool or function access | Weights emit a format; they cannot grant access | Agent orchestrator that routes the function calls |
| No held-out evaluation set | Cannot prove a gain or detect a regression | Build the eval set before touching weights |
Real products, models, and research that use this idea.
- Perplexity answers fresh web questions through retrieval over live results, not by retraining a model on the day's news, because weights would be stale within hours.
- OpenAI's fine-tuning guide explicitly recommends maxing out prompt engineering and few-shot examples before fine-tuning GPT-5.5 class models.
What an interviewer would ask next. Try answering before peeking at the approach.
QThe LIMA paper says 1000 curated examples beat large noisy sets, so why is under 500 a hard no?
Separate the quality claim from the volume claim. LIMA assumes a thousand genuinely hand-curated pairs, which is far harder to assemble than the few hundred noisy ones most low-data teams actually have.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Reaching for fine-tuning because the problem feels hard, instead of diagnosing whether it is a knowledge, format, or data problem that a cheaper tool solves better.
60 second bullets to scan on the way to the call.
Knowledge freshness as the first disqualifier for fine-tuning
Why retrieval beats retraining for changing facts
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.