Archive — US — techcrunch
Are bad incentives to blame for AI hallucinations?
A new research from OpenAI explores the persistent issue of AI hallucinations in large language models like GPT-5 and ChatGPT. Researchers attribute hallucinations to ineffective pretraining methods and suggest that current evaluation models create incentives for guessing rather than accuracy. They propose incorporating penalties for wrong answers to diminish the modeling behaviors that lead to hallucinations.
Key points
- Researchers define hallucinations as plausible but incorrect AI outputs.
- Example given: Incorrect responses to questions about an author's dissertation and birthday.
- Hallucinations occur due to a pretraining focus on predicting next words without true/false labels.
- Evaluation models incentivize incorrect predictions, encouraging guessing rather than uncertainty.
- Proposed solution: Update evaluation criteria to penalize guesses more heavily and reward expressions of uncertainty.
Why it matters
- AI hallucinations are a critical challenge that has implications for the reliability of generative AI systems.
- Current trends in AI revolve around enhancing accuracy and perception of AI outputs, particularly in regulated industries.
- Tech events like Disrupt 2025 emphasize the importance of addressing these challenges in AI development.
OpenAI's research underscores that to combat AI hallucinations effectively, evaluation methods must evolve. Shifting focus towards penalizing incorrect confident answers could lead to more reliable AI systems, making this an essential step as AI continues to integrate into various sectors.