Introduction to AI Chatbots and Mental Health
OpenAI, the company behind the popular chatbot ChatGPT, has been working to address concerns about the potential impact of its technology on mental health. Earlier this month, the company unveiled a wellness council to address these concerns, though critics noted the council did not include a suicide prevention expert. OpenAI also recently rolled out controls for parents of children who use ChatGPT.
Rare but Impactful Conversations
The data shared on Monday appears to be part of the company’s effort to demonstrate progress on these issues, although it also shines a spotlight on just how deeply AI chatbots may be affecting the health of the public at large. In a blog post on the recently released data, OpenAI says these types of conversations in ChatGPT that might trigger concerns about “psychosis, mania, or suicidal thinking” are “extremely rare,” and thus difficult to measure.
Statistics on Mental Health Conversations
The company estimates that around 0.07 percent of users active in a given week and 0.01 percent of messages indicate possible signs of mental health emergencies related to psychosis or mania. For emotional attachment, the company estimates around 0.15 percent of users active in a given week and 0.03 percent of messages indicate potentially heightened levels of emotional attachment to ChatGPT.
Efforts to Improve Safeguards
OpenAI also claims that on an evaluation of over 1,000 challenging mental health-related conversations, the new GPT-5 model was 92 percent compliant with its desired behaviors, compared to 27 percent for a previous GPT-5 model released on August 15. The company also says its latest version of GPT-5 holds up to OpenAI’s safeguards better in long conversations. OpenAI has previously admitted that its safeguards are less effective during extended conversations.
New Evaluations and Benchmarks
In addition, OpenAI says it’s adding new evaluations to attempt to measure some of the most serious mental health issues facing ChatGPT users. The company says its baseline safety testing for its AI language models will now include benchmarks for emotional reliance and non-suicidal mental health emergencies.
Changes to ChatGPT Content Restrictions
Despite the ongoing mental health concerns, OpenAI CEO Sam Altman announced on October 14 that the company will allow verified adult users to have erotic conversations with ChatGPT starting in December. The company had loosened ChatGPT content restrictions in February but then dramatically tightened them after the August lawsuit. Altman explained that OpenAI had made ChatGPT “pretty restrictive to make sure we were being careful with mental health issues” but acknowledged this approach made the chatbot “less useful/enjoyable to many users who had no mental health problems.”
Conclusion
OpenAI’s efforts to address mental health concerns related to ChatGPT are ongoing, and the company is taking steps to improve its safeguards and evaluations. However, the issue remains complex, and the company must balance the need to protect users’ mental health with the need to provide a useful and enjoyable experience.
FAQs
- Q: What is OpenAI doing to address mental health concerns related to ChatGPT?
A: OpenAI has established a wellness council, rolled out controls for parents, and is working to improve its safeguards and evaluations. - Q: How common are mental health emergencies related to ChatGPT?
A: According to OpenAI, these types of conversations are "extremely rare," with around 0.07 percent of users active in a given week and 0.01 percent of messages indicating possible signs of mental health emergencies related to psychosis or mania. - Q: What resources are available for users who may be experiencing mental health issues?
A: If you or someone you know is feeling suicidal or in distress, please call the Suicide Prevention Lifeline number, 1-800-273-TALK (8255), which will put you in touch with a local crisis center.
