Describe two automated hallucination detection techniques and their tradeoffs
Describe two automated techniques for detecting hallucinations in LLM outputs. For each, explain how it works and its main tradeoff.
NLI entailment checks each claim against a reference; self-consistency samples N responses and flags claims that vary. NLI needs a strong model; self-check misses confident systematic hallucinations.
Imagine a student writing an essay and you want to catch made-up facts. One way: take each sentence and check it against the textbook. If the textbook says nothing about it, or contradicts it, you flag it. That is the entailment approach, but it only works if you trust whoever is doing the checking. Another way: ask the student to write the same essay five times and see which facts stay the same. Facts that wobble from version to version are probably invented. But if the student always misremembers the same wrong date, every version agrees, and you never catch it. So one method needs a good reference and a sharp checker; the other needs no reference but goes blind whenever the student is confidently wrong the same way every time.
Detailed answer & concept explanation~8 min readEverything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Everything important, quickly.
5 min: NLI entailment mechanics and its model-quality cap, self-consistency sampling and its blind spot on confident errors, LLM-judge groundedness with citations, when each fits, and the precision versus recall trade.
Real products, models, and research that use this idea.
- RAGAS faithfulness decomposes the answer into claims and entailment-checks each against retrieved context rather than scoring holistically.
- SelfCheckGPT samples multiple generations and scores sentence-level consistency to flag hallucinations without any reference document.
- TruLens groundedness uses an LLM judge that must cite supporting source spans before scoring a claim as grounded.
- Patronus Lynx ships a fine-tuned open-weight hallucination detector tuned for entailment against context in RAG pipelines.
- DeepEval and Galileo expose claim-level faithfulness and consistency scores as first-class metrics in their 2026 eval dashboards.
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you set the entailment threshold differently for an auto-block queue versus a human review queue?
QWhy does self-consistency fail on confident systematic hallucinations, and what catches those instead?
Don't say thisRed flags and common mistakes that signal junior thinking. Click to expand.
Red flags and common mistakes that signal junior thinking. Click to expand.
Treating self-consistency as a truth check. It measures agreement, not correctness, so a confidently and repeatedly wrong fact passes every consistency vote undetected.
The night-before-the-interview bullets. Scan these on the way to the call.
Primary sources. Skim if you want the original framing.
Same topic, related formats. Practice these next.