A RAG product is shipping confident hallucinations whenever the retriever misses. Design an SFT recipe that teaches the model to refuse, i.e. say 'I don't have enough information in the provided context', when the retrieved chunks do not actually support an answer. Specify the data shape, how you construct positive vs negative examples, the balance between them, and at least one failure mode to watch.
Build matched (question, chunks, response) triples: half where chunks support the answer, half where chunks look related but do not, with templated refusals. Realistic negatives matter most.
Picture teaching a kid to say 'I don't know' when their notes do not cover a question. You quiz them with two kinds of cards. On half the cards, the notes do answer the question, and the kid is rewarded for using them. On the other half, the notes look related but actually do not answer it, and the kid is rewarded for saying 'these notes do not cover that'. The trick is the second pile has to look real. If you only quiz them with blank notes, they only learn to refuse when the page is empty, and they keep making things up whenever the page has any text on it. The realistic near-miss notes are what teach the actual signal.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Confident hallucinations in RAG products are typically a training-data problem, not a model-capacity problem. The base model was never explicitly taught that 'I don't have enough information' is sometimes the correct response. When the retriever misses and returns chunks that look related but do not actually answer the question, the model defaults to its prior of answering anyway, which produces fluent ungrounded text.
The fix is a targeted SFT recipe that teaches the model to refuse when context is insufficient. The recipe sounds simple, mix supported and unsupported examples, but the construction details determine whether the resulting model handles real production cases or only the easy ones.
This deep dive walks through the triple data shape, the construction of both halves of the matched populations, the three reliable techniques for sourcing realistic negatives, the templating discipline that makes the refusal cue clean, the partial-support slice that prevents brittle behaviour, and the precision-recall evaluation framework that exposes the cost of getting the trade-off wrong.
The triple data shape
Each training example is a triple: (question, retrieved_chunks, gold_response). The question is the user query. The retrieved chunks are what the retriever returned (or what you constructed to simulate a retrieval). The gold response is what the fine-tuned model should produce given the question and chunks.
This triple shape reflects the RAG inference flow exactly. At serving time, the retriever runs first and produces some chunks, which are concatenated with the user query into the model's prompt. The model generates a response conditioned on both. Training on triples that match this shape teaches the model the conditional behaviour you want.
The SFT loss masks the chunks and the question as conditioning, training only on the gold response tokens. This is standard chat-style masking: the model attends to the chunks and the question but is not scored on predicting them.
Format consistency between training and serving matters. If your serving prompt has a specific structure (e.g. 'Context: ...\n\nQuestion: ...\n\nAnswer:'), the training data must use the same structure. Mismatched formats break the learned behaviour because the model relies on the structural cues to know what counts as context, what counts as the question, and where its response should start.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Anthropic's Claude Opus 4.7 training reportedly includes grounded-refusal data for RAG and tool-use scenarios, with templated refusal phrasings.
- OpenAI's GPT-5.5 RAG fine-tuning guide recommends a similar matched-population recipe, with explicit examples of perturbed-query negatives.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you decide the precision-recall trade-off for refusal at evaluation time?
Quantify the production cost of false refusals (lost user trust, lost task completion) versus false answers (hallucinations shipped to users). Pick the operating point that minimises the weighted sum, and tune the negative-half size or training loss balance to land there.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Sourcing negative examples as blank or wildly off-topic context. The model then learns to refuse only on those obvious shapes and keeps hallucinating on the subtle near-miss cases that actually hurt in production.
60 second bullets to scan on the way to the call.
The triple data shape: question, retrieved chunks, gold response
The two matched populations: supported and unsupported
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.