Which of these are valid reasons NOT to fine-tune?
Skip fine-tuning when prompting or RAG already suffices, when knowledge changes often, when data is scarce, when you lack an eval harness, or when the base is already strong enough.
Fine-tuning is like sending an employee to an expensive multi-week training course. It permanently rewires how they work. You only do that when cheaper options have failed. If a sticky note on their desk fixes the problem, you write the note instead of booking the course. If the facts they need change every day, a note works but the course goes out of date instantly. If you have only three examples to train on, the course teaches the wrong habits. And if you cannot test whether the course actually helped, you have no way to know if you wasted the money. So before booking, you ask: would a note, a folder of references, or the employee's existing skills already do the job?
Detailed answer & concept explanation~7 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
4 min: escalation ladder framing + redundancy disqualifiers + missing-prerequisite disqualifiers + wrong-tool disqualifiers + the volume trap + eval harness as precondition.
| Situation | Reach for instead | Why not fine-tune |
|---|---|---|
| Knowledge updates daily | RAG over a live index | Weights go stale the moment data changes |
| Fewer than ~500 clean examples | Few-shot prompting | Training overfits instead of generalising |
| Prompt already clears the golden bar | Keep the prompt | Marginal lift, real regression risk |
| Need strict JSON output | Structured-output or constrained decoding | Format is decoding, not a weight edit |
| Need database or tool access | Tool-calling and orchestration | Access is infrastructure, not weights |
| High request volume | Maybe distillation for cost | Volume is not a quality gap |
Real products, models, and research that use this idea.
- OpenAI's own fine-tuning guide tells teams to exhaust prompt engineering and retrieval before fine-tuning, treating it as a later-stage optimisation.
- Teams enforcing strict JSON on GPT-5.5 or Claude Opus 4.7 use native structured-output modes and libraries like Outlines rather than fine-tuning for schema adherence.
- Perplexity keeps fast-changing web knowledge in a live RAG index rather than baking it into model weights that would go stale within hours.
- Distillation stacks fine-tune a small Llama 4 student on a larger teacher's outputs purely to cut per-call inference cost at high request volume.
- RAGAS and LangSmith eval harnesses are stood up first so teams can measure whether a fine-tune actually beat the prompt-only baseline.
What an interviewer would ask next. Try answering before peeking at the approach.
QWhy does high request volume motivate distillation but not plain fine-tuning?
QHow few examples is too few, and what actually goes wrong below that threshold?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Reaching for fine-tuning as the default lever. It is the expensive last resort. Prompting, RAG, and structured-output modes solve most problems first, and without an eval you cannot even tell if the fine-tune helped.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.