Flashcard: what does 'grounding' mean in the context of RAG, and why is it the whole point?

Attempt it

Grounding means the model's answer is constrained to be supported by the retrieved context, not by its parametric memory. A grounded answer cites specific chunks and refuses or hedges when the chunks don't cover the question. An ungrounded answer sounds confident but blends retrieved facts with what the model already 'knew' from pretraining, often inventing details (hallucination) and giving you no way to verify. Grounding is enforced via the system prompt ('answer only from the provided context, cite chunk numbers, refuse if insufficient') plus citation tagging. Without an explicit grounding directive, even good retrieval will not produce grounded answers; the model defaults to its broader knowledge.

Your answer

Quick reference

TL;DR

Grounding constrains the LLM to answer from the retrieved context only, with citations, and to refuse when the context does not cover the question. It is enforced by the system prompt.

Memory aid

Sign in for the mnemonic that makes this stick. See Pro pricing.

Easy to grasp

Imagine you ask a knowledgeable friend a question and also hand them a stack of documents. There are two ways they might answer. They could read the documents, point at the relevant paragraph, and tell you exactly what it says, refusing if the documents do not cover your question. Or they could glance at the documents, mix what they read with what they already think they know, and confidently make up a few details to round out the answer. The first version is grounded. The second is not. Grounding is the rule that says 'only use what is in the supplied documents, and cite where each fact came from'. Without that rule explicitly written into the prompt, even a smart model with great documents will default to the second behavior. That is the whole reason RAG bothers with system prompt directives.

Key concepts

RAG system designPrompt engineering

Detailed answer & concept explanation

~7 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Sign in for 5 free unlocks/day or go Pro for unlimited. See Pro pricing.

Quick scan · 2–4 min

Everything important, quickly.

3 min: define grounding as answer from context with citations and refusal, walk through the three system prompt directives, contrast with ungrounded behavior, and call out faithfulness as the production metric.

Real-world examples

Real products, models, and research that use this idea.

Anthropic's Claude with citations feature emitting per-claim references back to supplied chunks, with refusal when context does not support the answer.
Perplexity instructing its generator to cite every sentence with a numbered source, with a fallback message when sources are insufficient.
OpenAI's Responses API file-search returning citations metadata so the application can verify each generated claim against a specific chunk.
Ragas faithfulness scores being tracked as a per-release SLO inside enterprise RAG products to detect grounding regressions.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you write a system prompt that actually produces grounded answers?

Sign in or go Pro to see the approach for this follow-up.

QHow do you measure grounding in production, and what would a regression look like?

Sign in or go Pro to see the approach for this follow-up.

QWhy do small open weight models follow grounding instructions less reliably than frontier models?

Sign in or go Pro to see the approach for this follow-up.

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Assuming that retrieving relevant chunks is enough to make the answer grounded. Without an explicit grounding directive in the system prompt, the LLM blends retrieved context with parametric memory and produces fluent but ungrounded answers.

Sign in or go Pro to see all red flags and common mistakes. See Pro pricing.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

Sign in for 5 unlocks/day or go Pro for unlimited revision sheets. See Pro pricing.

References

Primary sources. Skim if you want the original framing.