Archive — US — techcrunch
OpenAI co-founder calls for AI labs to safety-test rival models
OpenAI and Anthropic have collaborated on joint safety testing of their AI models, amidst intense competition. Co-founder Wojciech Zaremba emphasized the need for industry-wide safety standards as AI enters a critical stage. The study revealed discrepancies in response reliability and raised concerns over AI 'sycophancy'. Both labs are optimistic about future collaborations despite competitive tensions.
Key points
- OpenAI and Anthropic temporarily opened their AI models for joint safety testing.
- This collaboration aims to identify weaknesses in their respective internal evaluations.
- The study found Anthropic's models were better at refusing uncertain answers, while OpenAI's showed higher hallucination rates.
- Concerns over AI safety heightened after a lawsuit linked ChatGPT to a tragic incident of a user's suicide.
- Both Zaremba and Anthropic's Carlini advocate for more collaborative safety testing in future AI developments.
Why it matters
- The collaboration reflects a growing trend in the tech industry for joint efforts on AI safety, critical as models become more integral to daily life.
- The focus on safety testing aligns with ongoing concerns about AI's societal impacts and ethical implications, echoing historical debates on technological responsibility.
- In 2025, AI labs are expected to prioritize partnerships to enhance innovation and regulatory compliance, shaping the future landscape of AI development.
The collaboration between OpenAI and Anthropic marks a significant step towards improved AI safety standards, illustrating the need for collective efforts in an increasingly competitive landscape as concerns over AI's societal impacts intensify.