Fine-tune for stable behavior at scale; prompt for fast iteration and changing tasks; use RAG for fresh or private knowledge.
Imagine you are training a new hire. You can write them a detailed instruction sheet (a prompt) that they read fresh every morning, or you can send them to a two-week bootcamp (fine-tuning) so the instructions become muscle memory. The sheet is flexible because you can rewrite it tonight, but the new hire has to re-read it every single day, which is slow at scale. The bootcamp is fast at runtime because they just know what to do, but updating their training means another bootcamp. And if the question is about facts they have not learned, neither helps. You hand them a reference book to look things up, which is the RAG pattern.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Prompt engineering and fine-tuning both adapt a base model toward a target behavior, but they sit at different layers in the stack and they have very different cost curves. A senior engineer keeps both in mind and reaches for whichever lever matches the actual gap. Reaching for the wrong one wastes weeks, and the wrong choice is usually fine-tuning when the real problem was missing facts or insufficient prompt iteration.
This question matters because the lever choice shapes the whole engineering plan. Fine-tuning means a data-curation team, a training pipeline, an eval harness for pre/post comparison, and a re-tune cadence. Prompt engineering means a versioned prompt library, a prompt-eval CI, and a faster but more frequent iteration loop. The teams look different, the budgets look different, and the failure modes look different.
The deep dive walks through how to think about the two levers, when each one wins, where they combine in production, and why the 2026 cost picture (LoRA, QLoRA, prompt caching) is different from the 2023 picture but the underlying decision rubric is mostly the same.
What each lever actually changes
Prompt engineering changes the input to a fixed function. The weights stay frozen. You are providing in-context instructions, examples, and constraints that condition the model's existing capabilities toward a specific behavior. Nothing about the model itself moves.
Fine-tuning changes the function. A training pass on a curated dataset updates the weights (full fine-tune) or a small set of low-rank adapters (LoRA, QLoRA). After the run, the model behaves differently even with no instructions, because the new behavior is now compiled into the parameters.
The practical consequence is that prompts are reversible in a minute and fine-tunes are not. A bad prompt change is a git revert. A bad fine-tune means retraining or rolling back to a previous adapter, which is hours of work and a new eval pass. This asymmetry is why production teams prompt first and fine-tune only when prompts have provably stopped moving the metric.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Concern | Prompt engineering | Fine-tuning |
|---|---|---|
| Iteration speed | Minutes; live edits and reverts | Hours to days; a training run per change |
| Cost per change | Near zero | $50-$500 with LoRA; more for full fine-tune |
| Best for | Task-level behavior, format guidance, reasoning scaffolds | Stable tone, strict schema, long stable instructions |
| Knowledge updates | RAG via retrieved context | Poor fit; weights do not reliably install facts |
| Production discipline | Versioned prompts + eval CI | Versioned adapters + golden-set evals pre/post |
Real products, models, and research that use this idea.
- Cursor ships hand-tuned system prompts against Claude Opus 4.7 and GPT-5.5, with no per-model fine-tune.
- Anthropic and OpenAI both offer LoRA fine-tuning on their flagship models, used mostly for strict output schemas and brand-tone consistency.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow does LoRA change the prompt vs fine-tune calculus in 2026?
Frame as a cost shift, not a behavior shift: training got cheaper, but data quality and eval discipline are still the gating constraints.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Reaching for fine-tuning when the gap is actually missing knowledge; that is a RAG problem, and weight updates will not reliably add facts.
60 second bullets to scan on the way to the call.
Three triggers that justify fine-tuning
Why knowledge gaps route to RAG
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.