Walk through how conflicting context arises and the context-layer rules that resolve it

Short answer·Medium·4.0 · 0·~3 min·Asked atAnthropic OpenAI Databricks

Attempt it

A context block contains two retrieved chunks that contradict each other on a policy question, plus a system rule that contradicts both. What does the model do, and what context-layer rules should you adopt to make outcomes predictable?

Free · 2 AI evals / day

**Conflicting context** is what happens when the prompt contains two or more sources that disagree about the same fact, and the model has to produce one answer. The most common shape in production is retrieval-heavy. The user asks a policy question, your retriever pulls back the top five chunks, and two of them contradict each other because the corpus contains both an old version of the policy and a new one. Worse, the system prompt has its own rule that contradicts both. Now the model is staring at three conflicting signals. The default behavior is bad. The model picks one source, usually the most recent in the prompt (because of the recency bias in attention), and answers confidently. The user has no idea there was a conflict, and the engineer has no idea why this answer came out instead of a different one. Three rules fix this. First, **precedence**. State in the system prompt which source wins when sources conflict, and tell the model to verbalize the conflict. Second, **sourcing**. Stamp every retrieved chunk with a label and a timestamp, and tell the model to prefer newer sources for the same topic. Third, **abstain**. If the rules do not resolve the conflict, the model should report the conflict back rather than guess.

**Conflicting context** is the situation where the prompt contains multiple sources that disagree on a fact the model must answer about. The default behavior, without intervention, is for the model to resolve the conflict invisibly: it picks one source, often the one in the recency slot of the prompt, and the user sees an answer that looks confident but is structurally arbitrary. Three context-layer rules make this predictable. **Precedence.** The system prompt should state explicitly which source wins. The typical hierarchy is: system rules first, tool outputs second, retrieved chunks third, user-provided documents fourth. The model should be instructed to verbalize when a conflict has been resolved by precedence, so the engineer can audit the resolution. **Sourcing.** Every retrieved chunk should carry a source label and a timestamp in the prompt itself. The model is told to prefer newer sources for the same topic, and to surface the source it used in the answer. This turns the conflict-resolution function from "trust the position in the prompt" into "trust the metadata," which is a property the engineer controls. **Abstain and report.** If precedence does not resolve the conflict and the sources are equally recent and equally trusted, the model should report the conflict to the user rather than pick. "Source A from 2024 says X; Source B from 2024 says Y; I cannot reliably choose. Please clarify." This is the honest answer, and it is far more useful than a confident wrong guess. The three rules together transform conflicting context from a silent failure into a visible signal. The engineer sees the conflict in the trace; the user sees it in the answer; the corpus owner gets a clear pointer to clean up the underlying data.

Quick reference

TL;DR

Without explicit precedence, sourcing, and abstain rules, the model silently resolves contradictions by position, producing answers that look confident but are arbitrary.

Memory aid

Easy to grasp

Imagine three friends giving you directions to the same restaurant, and each says something different. Without any rule for whom to trust, you just pick whoever spoke last and hope for the best. The model behaves the same way when retrieved chunks conflict: it picks one, often the most recent in the prompt, and answers as if it were certain. To make this safer, you give the model a small rulebook: trust the system message over retrieved text, trust newer sources over older ones on the same topic, and if there is no winner, say so out loud instead of guessing. Now disagreement becomes visible instead of being hidden inside a confident-sounding sentence.

Key concepts

Context Engineering RAG

Detailed answer & concept explanation

~8 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

Open with the default failure mode (silent, position-driven pick). Walk through the three rules: precedence with a concrete hierarchy, sourcing with the metadata it needs, abstain with the instruction it requires. Add the upstream fixes (rerank by authority, dedupe, route by partition). Close with the audit metric (order-stability rate) that detects silent failures.

Real-world examples

Real products, models, and research that use this idea.

Perplexity surfaces source citations inline and lets the user see when sources disagree
Glean enterprise search labels chunks with last-updated dates and routes queries by confidence
Notion AI uses workspace policies as system rules that override page content when they conflict
RAGAS evaluation framework includes a 'faithfulness' metric that catches silent conflict resolution
OpenAI's structured outputs make the conflict-resolution field a first-class part of the JSON schema

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow would you instrument detection of silent conflict-resolution failures in production?

QWhat happens when the system rule itself is the conflicting source?

QHow do constitutional or refusal-tuned models change the abstain calculus?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Assuming retrieved-context conflicts will resolve themselves and shipping without explicit precedence rules, then debugging arbitrary-looking answers in production.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.