Retrieved context changes the input the model conditions on, not its weights. Same fixed θ, different prompt, different output distribution. No parameter update happens at query time.
Think of the model as a sealed calculator. You can't open it up and rewire it, that's what training does, and it's already done. What you can do is type different numbers in. RAG doesn't rewire the calculator; it just types better numbers in by pasting relevant documents into the prompt. The calculator runs the exact same logic it always did, but because the input is now richer, the answer comes out different and more grounded. People sometimes say in-context learning 'teaches' the model on the fly. It doesn't. Nothing inside the box changes. The model is the same function before and after; only what you fed it changed. That distinction matters a lot when an interviewer asks what RAG actually does to the model.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
This question looks like a RAG question, but it's really probing whether you understand the boundary between what is fixed in a deployed model and what varies at inference. Many candidates can recite the RAG pipeline yet stumble the moment an interviewer asks what retrieval actually does to the model. The honest answer is: nothing, to the model itself.
A deployed large language model is a fixed mathematical function. Its parameters were frozen when training ended. Every clever thing RAG and in-context learning do happens on the input side of that function, never inside it. Getting this distinction crisp is what separates someone who memorized the slogan from someone who can debug a retrieval system and reason about when to fine-tune instead.
The model is a fixed function of θ
An autoregressive LLM is, mathematically, a single conditional probability distribution over next tokens. The whole model collapses to one expression, with the parameters appearing as a fixed argument:
The semicolon matters. Everything to the left of it is a variable you supply at inference, the prompt, plus the output y. Everything to the right, θ, is a parameter that was set during training and does not change when you call the model. The notation deliberately separates random variables from parameters: y and the prompt are quantities the distribution ranges over, while θ is a fixed setting that defines which distribution you have.
Think of it like a function in code whose constants are baked in at compile time. You pass arguments and get outputs, but you cannot reach in and rewrite the constants by calling the function. That is exactly the relationship between a query and the weights. Training is the compile step; inference is the call.
Concretely, θ is on the order of billions of floating point numbers sitting in GPU memory. A forward pass reads those numbers and multiplies them against the activations derived from your prompt. Nothing in that read and multiply loop writes back to θ. Backpropagation, the only thing that writes to θ, is a separate process that runs during training with a loss and an optimizer, none of which exist at inference time.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
| Aspect | RAG / in-context learning | Fine-tuning |
|---|---|---|
| What changes | The prompt (conditioning input) | The parameters θ |
| Persistence | Gone when the query ends | Durable across all future queries |
| Mechanism | Forward pass only, no gradients | Gradient descent on a training set |
| Math touched | Posterior P(y|prompt;θ) | θ itself (and thus the prior) |
| Best for | Fresh, private, query specific facts | Behavior, format, domain reasoning |
Real products, models, and research that use this idea.
- Perplexity feeds retrieved web pages into the prompt of a frozen model (GPT or Claude); same weights, grounded answers, zero retraining per query.
- Notion AI 2026 answers over your workspace by injecting retrieved docs into the context window of an unchanged base model.
What an interviewer would ask next. Try answering before peeking at the approach.
QIf no weights change, why does adding few-shot examples to the prompt reliably improve task accuracy?
Frame ICL as conditioning that lets the model infer a latent task variable from demonstrations; the forward pass routes activations differently without touching θ. Reference the Bayesian-inference view.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Saying RAG or in-context learning 'updates' or 'teaches' the model. The parameters never move at query time; only the conditioning input changes.
60 second bullets to scan on the way to the call.
Why a deployed LLM is a fixed function of θ
What the conditioning variable in P(y|prompt;θ) actually is
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.