Describe two automated hallucination detection techniques and their tradeoffs
Describe two automated techniques for detecting hallucinations in LLM outputs. For each, explain how it works and its main tradeoff.
NLI entailment checks each claim against a reference; self-consistency samples N responses and flags claims that vary. NLI needs a strong model; self-check misses confident systematic hallucinations.
Imagine a student writing an essay and you want to catch made-up facts. One way: take each sentence and check it against the textbook. If the textbook says nothing about it, or contradicts it, you flag it. That is the entailment approach, but it only works if you trust whoever is doing the checking. Another way: ask the student to write the same essay five times and see which facts stay the same. Facts that wobble from version to version are probably invented. But if the student always misremembers the same wrong date, every version agrees, and you never catch it. So one method needs a good reference and a sharp checker; the other needs no reference but goes blind whenever the student is confidently wrong the same way every time.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Hallucination detection asks a deceptively simple question: which claims in this answer cannot be supported? Framed that way, every detector is a binary classifier over claims, and the engineering reality is a precision versus recall trade rather than a single best method.
A crucial first move sits underneath all three techniques: claim decomposition. A model answer is rarely one fact; it is a paragraph that bundles several assertions, some grounded and some invented. You cannot label a paragraph as hallucinated or not, because parts of it are true. So every serious detector first splits the answer into atomic claims, single self-contained facts, and judges each one independently. The quality of that split bounds everything downstream, which is why it deserves its own attention before any classifier runs.
Three techniques then dominate the 2026 production stack. NLI-based entailment checks each claim against a reference. Self-consistency samples the answer many times and watches for claims that wobble. An LLM judge rates groundedness and must cite its evidence. This deep dive walks the mechanics of each, the exact failure mode that limits it, when it fits, and how teams layer the three to balance cost against catch rate.
NLI entailment: claim-level checking against a reference
The first technique treats hallucination detection as a Natural Language Inference problem. You decompose the generated answer into atomic claims, then run an NLI model on each claim paired with the reference context. The model returns one of three labels: entailed, neutral, or contradicted. Anything not entailed is flagged as a potential hallucination.
The appeal is cost and speed. A distilled NLI classifier runs in milliseconds per claim and scales to millions of claims a day. This is the engine behind RAG faithfulness in frameworks like RAGAS, where every sentence of the answer must trace back to the retrieved passages.
The tradeoff is that accuracy is capped by the NLI model. General-purpose NLI, trained on broad text, often fails on numeric reasoning, long-range coreference, and specialist jargon. On a medical or legal corpus it produces false negatives, marking unsupported claims as entailed because it does not understand the domain. The fix is a domain-tuned or instruction-tuned entailment model, plus careful claim decomposition, since a poorly split claim is unjudgeable.
from transformers import pipeline
nli = pipeline("text-classification", model="cross-encoder/nli-deberta-v3-large")
def flag_claim(claim: str, context: str) -> bool:
label = nli(f"{context} [SEP] {claim}", truncation=True)[0]["label"]
# ENTAILMENT -> grounded; CONTRADICTION / NEUTRAL -> flag
return label != "ENTAILMENT"Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- RAGAS faithfulness decomposes the answer into claims and entailment-checks each against retrieved context rather than scoring holistically.
- SelfCheckGPT samples multiple generations and scores sentence-level consistency to flag hallucinations without any reference document.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you set the entailment threshold differently for an auto-block queue versus a human review queue?
Auto-block punishes false positives, so tune for high precision with a strict threshold. A review queue tolerates false positives, so tune for high recall. Pick the operating point from the cost of each error type, not a default.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating self-consistency as a truth check. It measures agreement, not correctness, so a confidently and repeatedly wrong fact passes every consistency vote undetected.
60 second bullets to scan on the way to the call.
How NLI claim-level entailment classification works against a reference
Why NLI accuracy is capped by the NLI model on domain text
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.