Hallucination
Output from an AI system that is false or made up but stated with confidence, such as wrong facts, faulty reasoning or sources that don't exist.
Hallucinations are what make an AI system especially risky. Even if a hallucination is rare, it can jeopardize the safety of an entire system. Hallucinations are products of the probabilistic nature of models. Grounding in real sources, asking for citations and checking the output reduce them, and guardrails can catch some, but nothing removes them. Newer models hallucinate less on some tasks, but not on all of them.
Also called
- Confabulation
- Fabrication
- Making things up
Related
- AI slop Low-value, likely flawed AI output released without adequate checking by a person or a system.
- Sycophancy A model's tendency to tell you what you seem to want to hear.
How it connects
- Hallucination reduced by Grounding
- Probabilistic underlies Hallucination