Match each knowledge integration need to RAG, fine-tuning, or long context
Drag each answer to line up with its matching prompt
Large, frequently changing knowledge base needing citations
Fine-tuning
Teach a stable output style, format, or domain behavior
RAG (retrieve at query time)
A few documents supplied per request, used once
Fine-tuning (bake in the behavior)
Knowledge that updates hourly and must never be stale
Long context (put them in the prompt)
Consistent JSON schema / tone the base model keeps missing
RAG (fresh retrieval, no retraining)
Sort by what's changing: large or fast-changing knowledge that needs citations is RAG; stable behavior or format is fine-tuning; a few per-request docs is long-context.
Imagine a chef who needs help. If the ingredients change every day, you don't retrain the chef — you hand them today's fresh shopping list each time they cook. That's RAG. If the chef keeps plating messily, you train them once so good plating becomes a habit. That's fine-tuning. And if a single customer brings one special recipe card just for their meal, the chef just reads it that one time. That's long-context. The fork is simple: is the thing that's missing knowledge (and how often it changes), or a behavior the chef should always do?
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Matching needs to RAG, fine-tuning, or long-context is one of the most common LLM design questions, and it's usually framed as a rivalry between three techniques. That framing is misleading. They aren't competing answers to the same question — they solve genuinely different problems, and the 'matching' is really an act of diagnosis: name the gap, and the right tool follows.
The reason people get the pairs wrong is that all three can superficially make a model 'know more', so any of them feels like it could fit any item. But fine-tuning changes how the model behaves, RAG changes what facts it can reach at query time, and long-context changes what it sees in this one prompt. Those are three different verbs.
This deep dive builds the classifier that produces the matches reliably: first split knowledge gaps from behavior gaps, then split knowledge gaps by volume and freshness. We'll work each correct pair through that logic, show why the sibling distractors are tempting mismatches, and end on why real systems combine all three rather than picking one.
The first fork: knowledge gap or behavior gap?
Before you can match anything, name what the base model is missing. There are two fundamentally different kinds of gap.
A knowledge gap means the model doesn't have the facts — your company's policies, today's prices, a specific customer's history. The information exists; the model just can't reach it. The fix is to get the right facts in front of the model, either by retrieving them (RAG) or by supplying them in the prompt (long-context).
A behavior gap means the model knows plenty but acts wrong — it ignores your JSON schema, uses the wrong tone, or fumbles a domain task. No amount of facts fixes that; you have to change the model's default behavior. That's fine-tuning's job: it shifts the weights so the desired behavior becomes the model's habit.
This single fork already resolves several pairs. 'Teach a stable output style' and 'consistent JSON schema the base model keeps missing' are both behavior gaps, so both go to fine-tuning. 'Large changing knowledge base' and 'updates hourly' are both knowledge gaps, so neither is fine-tuning. Get this fork right and you've eliminated the most common mismatch — using fine-tuning to add facts — before you even look at volume or freshness.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Need | Best fit | Why | Main cost |
|---|---|---|---|
| Large, changing knowledge needing citations | RAG | Retrieve fresh chunks at query time, grounded in sources | Retrieval infra + latency |
| Stable output style / format / domain behavior | Fine-tuning | Bakes the behavior into weights | Training run per update |
| A few docs supplied per request, used once | Long-context | Just put them in the prompt; no infra | Tokens + latency scale with size |
Real products, models, and research that use this idea.
- Customer-support assistants use RAG over a hourly-updated knowledge base so answers cite current docs and never go stale.
- Teams fine-tune a model to reliably emit a strict JSON schema or house tone the base model keeps fumbling.
What an interviewer would ask next. Try answering before peeking at the approach.
QAt what point does a long-context approach need to become RAG?
When the material grows past what fits cheaply in the prompt, or is reused across many queries, so retrieving only the relevant chunks beats paying linear token cost every call.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Reaching for fine-tuning to add knowledge that changes often — it bakes a snapshot into weights and goes stale, exactly the job RAG's fresh retrieval is for.
60 second bullets to scan on the way to the call.
The two-step classifier: knowledge vs behavior, then volume vs freshness
Why 'updates hourly' is unambiguously a RAG case
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.