While OpenAI is building the latest and greatest AI tools for the masses, it is also busy tackling a lawsuit that holds ChatGPT culpable in a teen"s suicide. Apparently, the AI tool read the teen"s messages of distress and suicidal thoughts but rather than guiding him towards suicide prevention mechanisms, it offered to write a suicide note along with explaining ways to kill himself in a "beautiful suicide". Although the details of the case are quite horrifying and the fallout is yet to occur, OpenAI has now decided to implement some parental controls around ChatGPT.
In a blog post, OpenAI has announced that it is collaborating with the Expert Council on Well-Being and AI, which will help it to measure well-being that will guide product growth. In addition, it will be expanding its Global Physician Network to enhance its safety research and model training by gaining expertise in other areas like eating disorders, adolescent health, and substance abuse.
But perhaps, more important are the direct changes to ChatGPT. One of those includes a routing enhancement that will automatically direct a topic it identifies as containing signals of "acute distress" to GPT‑5-thinking so that it can generate more thoughtful responses.
In addition, OpenAI will begin rolling out parental controls in ChatGPT next month. Parents will be able to link to their teen"s account, configure how ChatGPT responds to their child through appropriate behavioral rules enabled by default, manage the disablement of features like chat history and memory, and receive a notification when the AI model detects a signal of "acute distress". OpenAI hasn"t defined the criteria for this notification system in detail, but has stated that guidance from experts will drive this process.
OpenAI has referred to young users of AI tools like ChatGPT as "AI natives", and noted that it will be sharing its progress updates in the area of improving its model to detect and respond to emotional distress throughout the next four months.