When an LLM in a RAG pipeline behaves differently because of the retrieved context, what is actually changing about the model?

An LLM is a fixed function once it's deployed. All its knowledge and behavior live in its parameters, usually written θ, and those numbers were frozen at the end of training. When you run **RAG**, you fetch relevant documents and paste them into the prompt before the model answers. The model then behaves differently, but not because anything inside it changed. The only thing that changed is the input you handed it. The clean way to say this: the model computes the probability of an output given the prompt and its weights. Adding retrieved context changes the prompt, so the output distribution shifts. The weights stay exactly where they were. This is why the correct answer is the one that says the model is the same function and only the input changed. The tempting wrong answers claim the parameters get updated, or that some special 'context' head fires, or that a Bayesian update rewrites the weights. None of that happens. RAG is just feeding the same model a better informed question.

An autoregressive LLM defines a single conditional distribution. The parameters θ are fixed at deployment: $$P(y \mid \text{prompt}; \theta)$$ Retrieving documents and inserting them into the prompt changes the conditioning variable. It changes the prompt, not the parameter θ. The same θ now produces a different output distribution because it's reading a different input. The model is the same function before and after; only what it conditions on changed. This is the heart of the distinction between **in-context learning** and fine-tuning. Fine-tuning runs gradient descent and physically moves θ, so the change persists across all future queries. In-context learning, which is what RAG exploits, moves nothing. When the query ends, the context is gone and the model is byte-for-byte identical to what it was before. The wrong answers each encode a popular misconception. One claims ICL performs real gradient steps on θ. Another invents a dedicated retrieval-aware attention head. A third says augmentation runs a Bayesian update on the weights. Augmentation does shift the posterior over outputs, but the prior was baked in by pretraining and the weights never move.