Hallucination
Also known as: Confabulation
When a model confidently makes up something that isn't true.
When a language model generates plausible-sounding but factually incorrect information. RAG, grounding, and constrained decoding are common mitigation strategies.
In practice
The headline failure mode for any production LLM app. Be ready to defend a mitigation stack (grounding, citation, eval, abstention) for your specific use case.
How it compares
Hallucination is the failure; RAG is one of the most common mitigations for it.
Comparisons that include Hallucination
Related topics
Questions that mention this term
- What does RAG primarily help with in LLM-based applications?
- Which metric best measures whether a RAG answer is grounded in the retrieved context?
- Design a fine-tune that teaches a model to refuse when the retrieved context lacks the answer.
- Describe a concrete, production runnable mechanism to detect hallucinations in a RAG answer (claims unsupported by the retrieved chunks), as the answer is generated or just after.
- TruthfulQA is designed to catch a specific model failure. Describe what it tests and why larger models sometimes do worse.
- Describe two automated hallucination detection techniques and their tradeoffs
Related terms
AI System Design
End-to-end design of production LLM systems: ingestion, retrieval, serving, eval, monitoring.
BM25
Classical TF-IDF-style lexical scoring: a surprisingly strong baseline, still the first stage of many hybrid RAG pipelines.
Chatbot Arena
Crowdsourced blind A/B comparisons of LLM responses, aggregated into an Elo leaderboard.
Chunking
Cutting documents into retrievable pieces before embedding them.
Context Engineering
Deciding what makes it into the model's context window (prompt + RAG + history + memory), and why.
Context Window
The max number of tokens a model can attend to at once.