ChatGPT rated "unacceptable risk" for teens after parental alerts failed during suicide conversations

ChatGPT rated "unacceptable risk" for teens after parental alerts failed during suicide conversations

The Common Sense Media Youth AI Safety Institute says OpenAI's safeguards for teenage ChatGPT users fall short in critical areas.

After more than 4,000 test prompts, the organization rated ChatGPT for Teens an "unacceptable risk" for minors and called for teenagers to be kept off the service until independent testing confirms it's safe.

The most damaging finding involves parental alerts. Across more than a dozen freshly created test accounts linked to parent accounts, explicit conversations about suicide, self-harm, and eating disorders never triggered a notification, The Verge reports. The results suggest that alerts only kick in after weeks of account history involving sensitive topics, not during acute crises. More than one in four situations that should have included a crisis referral failed to point users toward professional help.

The tutoring mode has gaps, too. Instead of guiding students through problems step by step, ChatGPT offered to display the finished answer right away. And despite an updated Under-18 Model Spec, ChatGPT kept responding in a friendly, personal tone when teens treated it like a person. Test accounts registered as adults never switched into teen mode either, even after several days and despite testers stating in the chat that they were 13 years old.Ad

OpenAI spokesperson Eric Porterfield disputed the findings, saying the tests didn't reflect how the safeguards work in practice. Tom Siegel of the institute countered that even accounts given enough time to activate the safeguards produced no notifications.

Parental alerts fail at the moment they matter most

OpenAI introduced parental controls specifically so parents would be notified during crises and built an entire Teen Safety Blueprint around that promise, which is now central to its defense in multiple lawsuits filed after documented teen deaths linked to ChatGPT.

The company's own data shows roughly two million people per week suffer psychological harm from the service, and its behavioral age prediction system, meant to catch minors on adult accounts, failed the most basic scenario in this test. With Florida already in court seeking to bar ChatGPT from minors entirely, an independent finding that the safeguards don't reliably work gives regulators and plaintiffs exactly the evidence they need.

These findings follow a pattern of real-world harm. A 16-year-old died after ChatGPT allegedly confirmed suicidal thoughts and provided concrete instructions, and a 23-year-old in Texas took his own life after ChatGPT reportedly responded to his suicidal ideation with approval for hours.Ad

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.