Flashcard: what does 'grounding' mean in the context of RAG, and why is it the whole point?

**Grounding** is the rule that the model's answer should come from the documents you retrieved, not from whatever the model happened to learn during pretraining. A grounded answer cites specific chunks. An ungrounded answer sounds confident and blends what was retrieved with what the model already 'knew', sometimes inventing details that are not in your documents at all. That invention is called hallucination, and grounding is the main thing that fights it. The surprising part is that grounding is not automatic. Even if your retriever returns perfect chunks, the LLM will not naturally restrict itself to them. By default, models answer from a blend of context and pretraining. You have to tell the model, explicitly in the system prompt, that it should answer only from the supplied context, that it should cite which chunk each claim came from, and that it should refuse or hedge when the context does not cover the question. Those three instructions are the enforcement mechanism for grounding. Skip them and your RAG answers will look authoritative while quietly lying.

**Grounding** in RAG is the property that an answer is anchored in the retrieved context, with explicit citations and explicit abstention when the context is insufficient. It is the whole point of the pattern. Without grounding, retrieval is wasted: the model still hallucinates, and the user has no way to tell which parts of the answer are supported. Three things separate a grounded RAG answer from an ungrounded one. A grounded answer uses only facts present in the retrieved chunks. A grounded answer attributes each claim to a specific chunk (chunk 2, chunk 5) so the answer is auditable. A grounded answer refuses or hedges when the retrieved context does not actually cover the user's question, instead of blending in parametric knowledge. The enforcement mechanism is the system prompt, not retrieval. Three directives must be explicit. The grounding directive: 'answer only from the supplied context'. The citation directive: 'cite the chunk number for each claim'. And the abstention directive: 'reply that the information is not available in the provided context if it is not'. Skip any of these and the model reverts to default behavior, which is to produce a fluent, confident, authoritative sounding answer that draws from pretraining whenever the context falls short. That silent failure mode is the most damaging RAG bug in production: nothing crashes, the system just lies smoothly.