Compare

In-context learning vs Fine-tuning

Adapt in the prompt, or adapt in the weights

The verdict

In-context learning adapts at inference by putting examples in the prompt; fine-tuning adapts by updating weights. Reach for prompting first; fine-tune when you cannot get there.

In-context learning

Glossary →

Teach the model at inference time by including examples in the prompt (few-shot) or a well-crafted instruction. No weight updates, no training run, no separate checkpoint.

Best for: Fast iteration and formats that keep changing.

Fine-tuning

Glossary →

Update the model's weights on a curated dataset. The learned behavior is baked into the checkpoint and does not have to be re-explained each request. LoRA and full fine-tuning are the common variants.

Best for: Stable formats at high query volume.

At a glance

In-context learning vs Fine-tuning: dimension-by-dimension comparison
Dimension	In-context learning	Fine-tuning
Where learning lives	Prompt (inference-time)	Weights (training-time)
Iteration speed	Minutes	Hours to days
Cost profile	Per-query input tokens	One-time train + serve
Ships in	Prompt template	Model checkpoint
Handles rare edge cases	Yes (add an example)	Only if in training data
Best for	Ambiguous format, iteration	Stable format, high volume

Key differences

1ICL happens at inference; fine-tuning happens at training time
2ICL costs per-query context tokens; fine-tuning costs a training run once
3Fine-tuning ships behavior in the checkpoint; ICL ships it in the prompt template
4Iteration on prompts is minutes; iteration on fine-tunes is hours to days
5At high volume, moving a stable pattern from prompt to weights saves a lot of tokens

In the interview

What they're really testing

Whether you default to ICL and can name when fine-tuning genuinely wins.

Say this

I default to in-context learning because iteration is minutes and there is no serving change. Fine-tuning earns its keep when a small number of formats are stable across many queries, when per-query token cost dominates, or when a small model must catch up to a big one on a narrow task. For most product features RAG plus few-shot beats fine-tuning on the same budget.

Traps to sidestep

Fine-tuning as a default reflex before trying prompting
Claiming fine-tuning teaches new facts (weights are unreliable for factual recall)
Ignoring that fine-tuning locks the format
Missing that ICL scales up to the context window ceiling

How to choose

If format is ambiguous or still changingIn-context learning

If same format across thousands of queries per dayFine-tuning

If per-query token cost dominatesFine-tuning

If small model must catch up to a big one on a narrow taskFine-tuning

Start with prompting. Fine-tune when the format is stable and per-query cost matters.

Common misconceptions

Myth: Fine-tuning is always better on domain data.

Reality: RAG plus few-shot usually beats fine-tuning on the same dataset and budget for factual recall. Fine-tuning wins on behavior and format, not on knowledge.

Myth: In-context learning does not scale.

Reality: It scales fine up to the context-window ceiling and iterates far faster than any fine-tune loop.

Memory aid

ICL is showing three examples before each task; fine-tuning is teaching until the model remembers.

Can you combine them?

The mature pattern is both: fine-tune for the behavior and format, and rely on ICL in the prompt for the last-mile edge cases and per-user personalization.