**Self-check consistency** detects hallucinations by sampling the same prompt several times and seeing whether the answers agree. The idea is intuitive: when a model is genuinely unsure, it makes things up differently each time, so the samples diverge. When it actually knows something, the samples stay stable. SelfCheckGPT is the best-known version of this technique.
The answer is the first option. Self-check measures inconsistency, not incorrectness. These are not the same thing.
The failure case is a confident, consistent error. Suppose a model has memorised a wrong historical date and reproduces it identically every time you ask. Every sample agrees, so the consistency check sees no wobble and reports no hallucination, even though the fact is plainly wrong.
The other options describe things that are not true. Self-check works on API-only models because it needs only the text outputs, not attention weights. It applies beyond facts to reasoning and code. And it typically uses 5 to 10 samples, not 100, so it is not prohibitively expensive.
The practical takeaway: pair self-check with a grounded method that checks claims against a trusted source, so consistent errors still get caught.
**Self-check consistency** samples a prompt multiple times and flags claims that disagree across samples. The premise is that fabricated content is unstable, so divergence signals a likely hallucination. SelfCheckGPT popularised this.
The core limitation is that it detects inconsistency, not incorrectness. A fact wrongly baked into the weights is reproduced identically across every sample. The check sees perfect agreement and passes the error through. So self-check has a systematic blind spot for confident, consistent mistakes.
The other options are wrong. It needs only text outputs, so it works fine against API-only models. It applies to reasoning and code, not just facts. And it uses roughly 5 to 10 samples, so cost is moderate, not prohibitive.
The practical fix is to combine methods. Self-check is a good cheap first pass that needs no reference. Pair it with a grounded check that compares claims against a trusted source. Entailment scoring or retrieval overlap catches the consistent errors that self-check misses. No single detector is sufficient alone.
Quick reference
TL;DR
Self-check measures inconsistency, not incorrectness. A confidently memorised wrong fact is reproduced identically across samples, so the consistency check passes and the error slips through.
Imagine checking a witness by asking the same question five times and trusting answers they repeat without wavering. This catches a witness who is guessing, because guesses change each time. But it completely fails on a witness who learned something wrong and is sure of it. They say the same wrong thing every single time, calmly and confidently. Your check sees perfect agreement and marks them reliable. Self-check consistency works exactly like this. It samples a model several times and flags claims that wobble between samples as likely fabricated. The blind spot is the confident, consistent mistake. If a wrong fact is baked deep into the weights, every sample agrees, the check passes, and the hallucination ships unnoticed.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Detecting factual errors the model holds confidently and reproduces every time Consistent samples pass the check, so the error is invisible. You need a grounded reference comparison instead.
Safety-critical or regulated outputs in medical, legal, or financial domains A reference-free consistency signal is too weak alone. Ground claims against a trusted source and add human review.
Strict latency or cost budgets on every single request Each self-check needs several extra generations. A single retrieval-overlap pass may be cheaper as a first filter.
Tasks with a clean ground-truth reference already available Direct entailment or exact match against the reference is more precise and avoids the consistent error blind spot entirely.
Quick scan · 2–4 min
Everything important, quickly.
4 min: why self-check detects inconsistency not incorrectness, the confident consistent error blind spot, and the layered detector stack (NLI, judge groundedness, retrieval overlap) that covers the gap.
Real-world examples
Real products, models, and research that use this idea.
1SelfCheckGPT pioneered reference-free, sampling-based hallucination scoring at the sentence level and remains the canonical self-check method.
2RAGAS faithfulness decomposes an answer into atomic claims and entailment-checks each against retrieved context, catching the consistent errors self-check misses.
3TruLens groundedness scores how well each statement in a RAG answer is supported by the source documents.
4Patronus and Galileo ship production hallucination detectors that layer entailment and LLM-judge groundedness rather than relying on self-consistency alone.
5Modern RAG stacks pair a cheap retrieval-overlap filter with a Claude Opus 4.7 or GPT-5.5 groundedness judge on flagged outputs.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow would you catch the confident, consistent hallucinations that self-check structurally misses?
Sign in or go Pro to see the approach for this follow-up.
QHow do NLI-based detection, LLM-judge groundedness, and retrieval overlap trade off against each other?
Sign in or go Pro to see the approach for this follow-up.
QHow would you design a tiered hallucination-detection pipeline for a production RAG system?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating self-check as a truth detector. It only measures whether samples agree, so a confident, consistent error passes the check while still being a hallucination.