You need an LLM app to answer questions about your company's internal docs, which are updated weekly. Should you RAG or fine-tune, and what factor most cleanly tips the decision?
The right answer is **RAG**, and the factor that tips it is how often the knowledge changes.
Think about what each approach does with facts. Fine-tuning bakes knowledge into the model's weights through a training run. RAG keeps the knowledge in a separate searchable index and looks it up at query time. The model itself is equally smart either way; only the location of the facts differs.
Now apply the weekly update detail. With fine-tuning, every time the docs change you have to run another training cycle. That is slow, costs money, and can make the model forget things it learned before. With RAG, you just re-embed the new or changed documents and add them to the index. That takes minutes and touches nothing else.
So for docs that update weekly, RAG wins on operational cost without giving up answer quality. The trap answer claims fine-tuned knowledge is deeper, but for looking up facts that is simply not true. The model reads the retrieved text and reasons over it the same way either way.
Choose **RAG**. The cleanly tipping factor is knowledge-change frequency, and weekly updating docs put this firmly in RAG territory.
The core distinction: RAG decouples the knowledge layer from the model. Your docs live in a vector index; you retrieve relevant chunks at query time and feed them to the LLM as context. Fine-tuning instead encodes knowledge into the model's parameters during a training run.
That decoupling is exactly what the weekly cadence rewards. Updating a RAG index is incremental and cheap: re-embed the changed documents, upsert them, done in minutes. Re-fine-tuning is the opposite: a full training cycle each week, with real GPU cost, latency in hours, and the risk of catastrophic forgetting where new training degrades previously learned behavior.
The tempting wrong answer is that fine-tuning gives 'deeper' knowledge. For factual recall that is empirically false; the LLM synthesizes over retrieved context with the same reasoning capacity. Fine-tuning earns its keep for behavior: output format, domain voice, tool-call structure.
Hybrid (fine-tune the voice, RAG the facts) is sometimes the production answer, but the question asks for the single most-cleanly tipping factor, and that is update frequency.
Quick reference
TL;DR
RAG. The docs change weekly, so the cleanly tipping factor is knowledge-change frequency: re-indexing is cheap and incremental, while re-fine-tuning weekly is slow, costly, and risks forgetting.
Imagine you run a help desk and the rulebook gets a new page every week. One option: make each agent memorize the whole rulebook again every Monday. That is fine-tuning. It is slow, expensive, and they sometimes forget old rules while cramming new ones. The other option: keep the rulebook on a shelf and let agents look things up when a customer asks. When a page changes, you just swap that one page on the shelf. That is RAG. The agents (the LLM) are just as smart either way; the only difference is where the facts live. When the facts change often, you want them on a shelf you can edit in seconds, not memorized in heads you have to re-train. So weekly changing docs point straight at RAG.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
The customization needed is behavior, not facts: a fixed output format or domain voice Behavior is diffuse and process shaped; retrieval can't encode it, so fine-tuning is the right tool, not RAG.
The internal docs are tiny and effectively frozen: a few pages that never change If everything fits in the prompt and never updates, retrieval machinery is overhead with no freshness payoff.
Knowledge must be consistent to the second across a strict consistency system Vector index lag can serve stale chunks; a live data call or write through indexing may beat batch re-indexing.
Quick scan · 2–4 min
Everything important, quickly.
3 min: name RAG, state knowledge-change frequency as the tipping factor, contrast re-index vs re-train cost, bust the 'deeper knowledge' myth, then note hybrid as additive.
Side-by-side
Concern
RAG
Fine-tuning
Knowledge-change frequency (the tipping factor)
Incremental re-index in minutes; built for frequent change
Full retraining cycle per update; punishing for weekly cadence
Update cost / turnaround
Cheap; re-embed the diff and upsert
GPU spend + hours per cycle + eval gate
Factual depth / recall
Strong; surfaces verbatim source with citations
Lossy parametric memory; hallucinates on the long tail
Behavior (format, voice, tools)
Prompt-level only, limited
Strong; baked into weights
Per-query latency
Higher; retrieval + extra context tokens
Lower; no retrieval step
Regression risk
Index change is isolated and reversible
Catastrophic forgetting of prior knowledge
Real-world examples
Real products, models, and research that use this idea.
1Glean indexes a company's Slack, Drive, and Confluence into a RAG layer that re-syncs as docs change, instead of fine-tuning per customer.
2Notion AI runs RAG over your workspace so edits made today are answerable today, with no retraining step.
3GitHub Copilot Chat retrieves from your current repo and open files rather than fine-tuning on each evolving codebase.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QSuppose the docs updated once a year instead of weekly; does your answer change, and why?
Sign in or go Pro to see the approach for this follow-up.
QWhat's catastrophic forgetting and how does it bite a weekly fine-tuning loop?
Sign in or go Pro to see the approach for this follow-up.
QWhen would you actually combine fine-tuning with RAG for this internal docs app?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Assuming fine-tuned knowledge is 'deeper' than retrieved knowledge. For factual recall it is not; and repeated fine-tuning weekly is the operational killer here, regardless of depth.