Archive — US — techcrunch
OpenAI’s new reasoning AI models hallucinate more
OpenAI's new reasoning AI models, o3 and o4-mini, are showing increased hallucination rates compared to their predecessors, raising concerns about accuracy. Despite improvements in coding and math performance, the models hallucinate 33% and 48% of the time respectively on PersonQA. OpenAI acknowledges a lack of understanding as to why this is happening, highlighting a need for further research.
Key points
- OpenAI launched o3 and o4-mini models, which are advanced but hallucinate more than older models.
- Historically, newer models have shown slight improvements in reducing hallucinations, which is not the case here.
- On PersonQA benchmark: o3 hallucinated 33% of the time, o4-mini 48%, significantly more than previous models.
- Issues may stem from the reinforcement learning methods used in training, potentially amplifying hallucinations.
- Despite these issues, o3 shows superior performance in coding tasks but tends to generate broken links.
- Hallucinations complicate the use of AI in fields demanding high accuracy, such as law.
- Implementing web search capabilities in AI models may help reduce hallucinations.
Why it matters
- The AI industry is increasingly moving toward reasoning models, relying on new techniques as older ones show diminishing returns.
- Hallucination remains a significant problem across various AI applications, impacting their reliability and applicability.
- Advancements in AI are also exploring the integration of real-time data access to enhance accuracy and mitigate hallucination.
The rising hallucination rates in OpenAI's o3 and o4-mini models highlight an urgent need to address this ongoing challenge, especially as the industry shifts towards reasoning models for improved performance.