EXPLORINEER
en

Archive — US — techcrunch

Google Gemini dubbed ‘high risk’ for kids and teens in new safety assessment

Common Sense Media assessed Google's Gemini AI as 'high risk' for kids, citing concerns over inappropriate content and insufficient safety features. While it properly indicated its identity as AI, it failed to tailor guidance for different age groups. Google refuted the claims, stating it has protections for minors but acknowledged some responses were faulty. The ongoing debate raises concerns about AI's impact on youth safety amid rising risks of mental health issues.

Key points

  • Common Sense Media released a safety assessment on Google Gemini AI, labeling it as 'high risk' for children and teens.
  • The AI clearly identified itself as a computer, which helps reduce the risk of delusional thinking.
  • However, Gemini's content still poses risks, sharing inappropriate information about sex, drugs, and mental health.
  • Safety features lack customized content for different age groups, primarily reflecting adult versions.
  • Common Sense believes AI should be designed with children’s safety in mind from the ground up.
  • The assessment comes amid reports of AI's role in teen suicides, notably related to ChatGPT and Character.AI.
  • Google pushed back against the assessment, affirming it has safety policies for under-18 users and is improving its safeguards.

Why it matters

  • There is increasing scrutiny over AI's role in children's mental health and safety, especially as incidents of suicide linked to AI interactions rise.
  • Previous assessments of similar AI products by Common Sense Media indicate a broader trend of evaluating AI safety for minors.
  • With tech companies focusing on developing AI for various applications, including Apple's interest in Gemini for Siri, the safety concerns are critical for public discussions in the tech industry.

The evaluation of Google Gemini highlights essential concerns about the safety of AI for young users. As discussions around youth mental health and AI ethics continue, it emphasizes the need for well-designed safeguards tailored to children's developmental needs.