Give three concrete situations where fine-tuning is the wrong tool. For each, explain why FT specifically fails and what to reach for instead.
Free · 2 AI evals / day
Fine-tuning sounds like the serious fix for a struggling model, so people reach for it too early. Often a cheaper tool is the correct answer, and fine-tuning would waste time or make things worse.
The first wrong case is fresh knowledge. Fine-tuning bakes facts into the weights at training time. If your data changes weekly, the model goes stale fast, and you cannot retrain every week. Retrieval is the fix: keep the model frozen and feed it the latest documents at query time.
The second wrong case is when a good prompt already works. If a few-shot prompt hits 85 percent on your test set, fine-tuning is low-leverage. It costs effort and can quietly hurt other skills. Improve the prompt or examples first.
The third wrong case is too little data. With under 500 clean examples, fine-tuning often under-converges and lands below the base model with a good prompt. Gather more data or just use the best examples as in-context examples instead.
A fourth trap: you also need an evaluation set. Without one, you cannot prove the fine-tune helped, so you may ship a regression blind.
Fine-tuning is the wrong tool whenever the problem is not actually about model behavior. Three concrete cases come up constantly.
First, frequently-updated knowledge. Fine-tuning freezes facts into weights at train time. When the source data changes, the model is stale, and retraining on every change is absurd. Reach for RAG: index the corpus and retrieve at query time, so updates are instant.
Second, a prompt that already clears the bar. If a few-shot prompt scores above 85 percent on your golden set, fine-tuning is a low-leverage move. You risk regressions on other tasks for a small last-mile gain. Iterate on prompts, examples, or structured-output mode first.
Third, too few examples. Below roughly 500 clean pairs, fine-tuning under-converges. LoRA on 200 examples often loses to the base model with a strong prompt. Gather more data or fall back to in-context learning.
A fourth gate cuts across all three. If you have no evaluation set, do not fine-tune at all. You cannot prove the change helped, and a silent regression is worse than the original gap. Build the eval before touching weights.
Quick reference
TL;DR
Don't fine-tune when prompting already works, when knowledge changes often, when you have under 500 examples, or when there is no eval to prove it helped.
Fine-tuning is like sending a chef to a months-long cooking school to permanently change how they cook. That makes sense if you want a new style of food forever. But it is the wrong move for some problems. If today's menu changes daily, you do not retrain the chef each morning: you just hand them today's menu, which is retrieval. If the chef already cooks the dish well from a written recipe, retraining is wasted money and might make their other dishes worse. And if you only have three example plates to learn from, the school has nothing solid to teach from. So before retraining, ask: is this really a cooking-habit problem, or just a missing-menu problem?
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Answering over a knowledge base that updates every few days or hours Fine-tuning freezes facts at train time, so the model is stale fast. Retrieval keeps the index live and current.
A few-shot prompt that already scores above 85 percent on the golden set The gain is marginal and the regression risk is real. Cheaper prompt iteration usually closes the last-mile gap.
Fewer than 500 high-quality prompt and response pairs on hand The run under-converges and can land below base plus a strong prompt. In-context learning is the safer fallback.
No held-out evaluation set to measure the change against Without an eval you cannot tell a real gain from a confident regression that ships silently.
Wanting the model to gain access to a new tool or API Access is routed by the agent orchestrator. Fine-tuning teaches the call format, not the access itself.
Quick scan · 2–4 min
Everything important, quickly.
4 min: diagnose problem type + knowledge freshness goes to RAG + prompt already works regression trade + under 500 examples floor + missing eval gate + tool access is infrastructure.
Side-by-side
Symptom
Why fine-tuning fails
Reach for instead
Knowledge updates weekly or faster
Facts freeze in weights at train time and go stale
RAG with a live index, retrieved at query time
Few-shot prompt already above 85 percent
Low leverage plus regression risk for a small gain
In-context learning, or gather and curate more data
Need tool or function access
Weights emit a format; they cannot grant access
Agent orchestrator that routes the function calls
No held-out evaluation set
Cannot prove a gain or detect a regression
Build the eval set before touching weights
Real-world examples
Real products, models, and research that use this idea.
1Perplexity answers fresh web questions through retrieval over live results, not by retraining a model on the day's news, because weights would be stale within hours.
2OpenAI's fine-tuning guide explicitly recommends maxing out prompt engineering and few-shot examples before fine-tuning GPT-5.5 class models.
3Teams adopting structured-output and JSON-schema modes on Claude Opus 4.7 and Gemini 3.1 Pro get format adherence with zero training, sidestepping a fine-tune.
4The LIMA study showed 1000 curated examples can rival large noisy sets, but most low-data teams fall well under that floor and do better with in-context learning.
5Enterprises under data-residency rules keep customer records in a Pinecone or Weaviate index for RAG rather than baking them into a Llama 4 fine-tune.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QThe LIMA paper says 1000 curated examples beat large noisy sets, so why is under 500 a hard no?
Sign in or go Pro to see the approach for this follow-up.
QIf structured output mode fixes JSON adherence, when is fine-tuning the format ever justified?
Sign in or go Pro to see the approach for this follow-up.
QHow would you set the bar that prompt engineering has truly plateaued before committing to a fine-tune?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Reaching for fine-tuning because the problem feels hard, instead of diagnosing whether it is a knowledge, format, or data problem that a cheaper tool solves better.