What is the main limitation of self-check consistency as a hallucination detection technique?

**Self-check consistency** detects hallucinations by sampling the same prompt several times and seeing whether the answers agree. The idea is intuitive: when a model is genuinely unsure, it makes things up differently each time, so the samples diverge. When it actually knows something, the samples stay stable. SelfCheckGPT is the best-known version of this technique. The answer is the first option. Self-check measures inconsistency, not incorrectness. These are not the same thing. The failure case is a confident, consistent error. Suppose a model has memorised a wrong historical date and reproduces it identically every time you ask. Every sample agrees, so the consistency check sees no wobble and reports no hallucination, even though the fact is plainly wrong. The other options describe things that are not true. Self-check works on API-only models because it needs only the text outputs, not attention weights. It applies beyond facts to reasoning and code. And it typically uses 5 to 10 samples, not 100, so it is not prohibitively expensive. The practical takeaway: pair self-check with a grounded method that checks claims against a trusted source, so consistent errors still get caught.

**Self-check consistency** samples a prompt multiple times and flags claims that disagree across samples. The premise is that fabricated content is unstable, so divergence signals a likely hallucination. SelfCheckGPT popularised this. The core limitation is that it detects inconsistency, not incorrectness. A fact wrongly baked into the weights is reproduced identically across every sample. The check sees perfect agreement and passes the error through. So self-check has a systematic blind spot for confident, consistent mistakes. The other options are wrong. It needs only text outputs, so it works fine against API-only models. It applies to reasoning and code, not just facts. And it uses roughly 5 to 10 samples, so cost is moderate, not prohibitive. The practical fix is to combine methods. Self-check is a good cheap first pass that needs no reference. Pair it with a grounded check that compares claims against a trusted source. Entailment scoring or retrieval overlap catches the consistent errors that self-check misses. No single detector is sufficient alone.