Retrieval surfaces chunks that directly contradict each other, for example an outdated policy and the current one, or two specs that disagree. Explain how a production RAG system should handle the conflict and why silently picking one answer is the wrong default.
Free · 2 AI evals / day
Retrieval often pulls back chunks that disagree. One chunk quotes the old policy, another quotes the current one. Two product specs list different limits. The dangerous default is for the model to quietly pick one and present it as fact, because the user has no way to know a second source said the opposite.
The right behavior depends on whether the system has a signal to break the tie. The first good move is to resolve by **recency**: if the conflict is a versioning issue, prefer the document with the newer effective date or timestamp. The second is to resolve by **authority**: if some sources outrank others, prefer the official spec over a forum post.
When neither signal cleanly decides, do not guess. Surface the conflict: say the sources disagree, show each answer, and cite both so the user can judge. That turns a silent failure into an honest one.
All of this depends on groundwork at ingestion. The chunks must carry recency and authority as metadata, and citations must be tracked through to generation. Without that, the system has nothing to reason with and falls back to a coin flip dressed up as confidence.
A production RAG system must treat contradicting chunks as a first-class case, not let the language model resolve them by accident. The failure mode to avoid is **silent collapse**: the model blends or favors one chunk and emits a single confident answer. Whichever side it picked may be wrong, and the user cannot tell, so the system has manufactured a confidently wrong response.
There are two principled resolution strategies. Resolve by recency when the conflict is a versioning issue — prefer the document with the newer effective date or last-updated timestamp, since the old policy is simply stale. Resolve by authority when sources have a clear ranking — an official spec outranks a community post, a signed contract outranks a draft. These rules only fire when the metadata supports them.
When no signal cleanly decides, the correct move is to surface the disagreement. State that sources conflict, present each position, and cite both so the user can adjudicate. Abstaining from a false certainty is better than guessing.
None of this works without upstream investment. Ingestion has to capture recency and authority as structured metadata on every chunk, and citation tracking has to thread source identity from retrieval through to the generated answer. The system can only reason about a conflict it can see, and it can only let the user judge a source it can name.
Quick reference
TL;DR
When retrieved chunks contradict, resolve by recency or authority metadata where a signal cleanly decides, and otherwise surface the conflict with citations — never silently pick one side as confident fact.
Imagine you ask two coworkers what the refund window is, and one says 30 days while the other says 14. A bad assistant just blurts out one number and sounds sure. A good assistant checks who is more trustworthy — maybe one is quoting last year's handbook and the other the current one — and if it still cannot tell, it says, 'These two notes disagree: one says 30 days, the other 14, here is where each came from.' That is what a careful RAG system does: it uses dates and source rank to break the tie, and when nothing breaks it, it shows you both and lets you decide instead of guessing for you.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Single-source corpora where one authoritative document is the only ground truth If the index only ever holds the current canonical version, conflicts cannot arise and the resolution machinery is dead weight.
Creative or opinion tasks where multiple views are the point Surfacing disagreement is the wrong frame when the user wants synthesis or brainstorming; the contradiction is a feature, not a fault.
Latency-critical lookups with no metadata to reason over Adding a detection and resolution pass costs an extra call; if ingestion never captured recency or authority, the rules have nothing to read and only add delay.
Quick scan · 2–4 min
Everything important, quickly.
Open with why silent collapse is the cardinal sin: it manufactures confident wrongness the user cannot detect. Lay out the decision tree — detect the conflict, attempt deterministic resolution by recency then authority, and escalate to surfacing both cited sources when nothing decides. Then explain the unglamorous prerequisites: ingestion must capture recency and authority metadata, supersession must demote stale docs, and citation identity must survive chunking and reranking. Close on treating conflict handling as a measurable faithfulness target in the eval harness.
Real-world examples
Real products, models, and research that use this idea.
1Perplexity surfaces multiple cited sources rather than collapsing conflicting search results into one unattributed claim, letting the reader weigh disagreements
2Enterprise policy assistants prefer the document with the newest effective date when an old HR policy and its revision both get retrieved
3Legal and compliance RAG tools rank an official statute above a secondary commentary when the two disagree on a requirement
4RAGAS faithfulness scoring flags answers whose claims are not jointly entailed by the retrieved context, catching masked contradictions in eval
5Customer-support bots that quote a superseded pricing page erode trust the moment a user opens the current page and sees a different number
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you automatically detect that two retrieved chunks contradict each other before generation?
Sign in or go Pro to see the approach for this follow-up.
QWhat breaks your recency rule when documents are versioned but the dates are unreliable?
Sign in or go Pro to see the approach for this follow-up.
QHow do you keep a superseded document from ever winning a conflict in the first place?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Letting the model silently favor whichever contradicting chunk landed first in the context, producing a confidently wrong answer that the user only catches when they read the other source.