Three concrete situations where fine-tuning is the wrong tool

Fine-tuning sounds like the serious fix for a struggling model, so people reach for it too early. Often a cheaper tool is the correct answer, and fine-tuning would waste time or make things worse. The first wrong case is fresh knowledge. Fine-tuning bakes facts into the weights at training time. If your data changes weekly, the model goes stale fast, and you cannot retrain every week. Retrieval is the fix: keep the model frozen and feed it the latest documents at query time. The second wrong case is when a good prompt already works. If a few-shot prompt hits 85 percent on your test set, fine-tuning is low-leverage. It costs effort and can quietly hurt other skills. Improve the prompt or examples first. The third wrong case is too little data. With under 500 clean examples, fine-tuning often under-converges and lands below the base model with a good prompt. Gather more data or just use the best examples as in-context examples instead. A fourth trap: you also need an evaluation set. Without one, you cannot prove the fine-tune helped, so you may ship a regression blind.

Fine-tuning is the wrong tool whenever the problem is not actually about model behavior. Three concrete cases come up constantly. First, frequently-updated knowledge. Fine-tuning freezes facts into weights at train time. When the source data changes, the model is stale, and retraining on every change is absurd. Reach for RAG: index the corpus and retrieve at query time, so updates are instant. Second, a prompt that already clears the bar. If a few-shot prompt scores above 85 percent on your golden set, fine-tuning is a low-leverage move. You risk regressions on other tasks for a small last-mile gain. Iterate on prompts, examples, or structured-output mode first. Third, too few examples. Below roughly 500 clean pairs, fine-tuning under-converges. LoRA on 200 examples often loses to the base model with a strong prompt. Gather more data or fall back to in-context learning. A fourth gate cuts across all three. If you have no evaluation set, do not fine-tune at all. You cannot prove the change helped, and a silent regression is worse than the original gap. Build the eval before touching weights.