Citation presence shows the model learned to reference sources but does not prove the cited content supports the claim. Faithfulness metrics verify claim level entailment.
Imagine a student who writes a history essay and puts footnotes at the end of every sentence. The teacher is impressed until she checks the footnotes. One footnote points to a real book but gets the date wrong. Another points to a book that does not exist. The student learned that essays should have footnotes, but the footnotes do not actually back up the claims. A grounded model is like a student whose footnotes actually say what the essay claims they say. Checking that takes more work than counting footnotes.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
The claim that a model is grounded because it cites sources confuses a formatting behavior with a semantic property. Citation presence means the model learned that certain contexts predict citation tokens. Factual grounding means the content of the output is actually supported by the cited source. These are independent properties, and measuring one tells you nothing about the other.
This distinction matters most in retrieval augmented generation systems, where the model has access to source documents and is expected to ground its answers in them. The failure mode is subtle: the output looks well referenced, the citations point to real documents, but the claims do not match what the documents actually say. Catching this requires a different class of metric than citation counting.
How models fabricate citations
Models generate citations the same way they generate any other token: by predicting what is statistically likely given the context. There is no internal verification step where the model retrieves the source, reads it, and confirms the claim before emitting the citation token.
This produces two fabrication patterns. In the first, the model cites a real paper but attributes the wrong finding to it. The paper exists, the authors are correct, the year is right, but the result described in the output is not what the paper actually found. This is the harder failure to catch because surface level validation (checking the URL, confirming the paper exists) passes.
In the second pattern, the model cites a paper that does not exist at all. It assembles a plausible title, a plausible author list, and a plausible venue from distributional patterns in training data. The citation looks real but points to nothing. Studies on long form generation have found fabrication rates between 10 and 40 percent depending on the model and task domain.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS (Retrieval Augmented Generation Assessment) is widely used in production RAG pipelines to score faithfulness by decomposing outputs into claims and checking entailment against retrieved context.
- FActScore evaluates long form generation by breaking output into atomic facts and verifying each against a knowledge source, catching misattributed citations that surface level checks miss.
What an interviewer would ask next. Try answering before peeking at the approach.
QYour RAG system scores 0.92 on RAGAS faithfulness but users still report hallucinations. What could explain the gap?
RAGAS faithfulness measures entailment against retrieved context, not against ground truth. If the retrieved context itself is wrong or incomplete, the model can faithfully reproduce incorrect information and score high on faithfulness. You need a second metric that checks the retrieved context against a verified source, or use FActScore with a reliable knowledge base as the reference.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Equating citation count with grounding quality, when models routinely fabricate references or misattribute findings to real papers.
60 second bullets to scan on the way to the call.
Why citation presence does not prove factual grounding.
Two modes of citation fabrication: wrong findings attributed to real papers, and entirely fabricated references.
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.