OpenAI adds Trusted Contact feature to alert emergency contacts when ChatGPT detects self-harm discussions
OpenAI launched an optional Trusted Contact feature for ChatGPT that notifies designated emergency contacts when the system detects discussions about self-harm or suicide. The feature requires manual review by trained personnel before sending notifications, and does not share chat transcripts with contacts.
ChatGPT Adds Emergency Contact Alerts for Self-Harm Detection
OpenAI launched an opt-in safety feature that allows ChatGPT users to designate a "Trusted Contact" who will be notified if automated systems detect discussions about self-harm or suicide with the chatbot.
The feature is available to all adult users globally (18+ or 19+ in South Korea). Users can add contact information for one person through their ChatGPT account settings. The designated contact must accept the invitation within one week.
How the System Works
When OpenAI's automated systems flag a conversation indicating potential self-harm, ChatGPT prompts the user to contact their Trusted Contact. A small team of trained personnel then manually reviews the flagged conversation. If the review confirms serious safety concerns, the system sends a notification via email, text message, or in-app ChatGPT alert.
According to OpenAI, notifications are "intentionally limited" and do not include chat details or transcripts. Either party can remove themselves from the arrangement at any time through account settings.
Background and Context
The feature expands parental controls introduced in September 2024, which followed the suicide of a 16-year-old who had spent months confiding in ChatGPT. OpenAI already provides localized crisis helpline information within ChatGPT responses.
Meta deployed a similar feature on Instagram that alerts parents when teenagers repeatedly search for self-harm content. OpenAI previously faced criticism after reports that ChatGPT responses may have reinforced delusional thinking in some users experiencing mental health crises.
What This Means
This represents a shift in how AI companies handle duty-of-care responsibilities for conversational AI. By inserting human review between automated detection and notification, OpenAI acknowledges that pure algorithmic approaches to crisis intervention carry significant false positive risks. The feature's opt-in nature sidesteps consent issues while addressing concerns that ChatGPT functions as an unmonitored confidant for vulnerable users. However, the effectiveness depends on detection accuracy and whether users at risk will proactively enable the feature.
Related Articles
U.S. government orders Anthropic to halt exports of Mythos and Fable AI models, both now offline for one week
The White House ordered Anthropic to restrict exports of its Mythos and Fable AI models last Friday, citing national security concerns. Anthropic pulled both models offline within 90 minutes of the Commerce Department directive, marking the first major test of AI export controls.
GitHub details Qubot, internal Copilot-powered data analytics agent for plain language queries
GitHub has released technical details on Qubot, an internal analytics agent powered by GitHub Copilot that enables employees to query company data using natural language. The agent represents GitHub's implementation of AI-assisted data analysis for internal operations.
GitHub built Qubot, an internal data analytics agent using Copilot to query company data in natural language
GitHub has built Qubot, an internal analytics agent powered by GitHub Copilot that allows employees to query company data using natural language. The project represents GitHub's approach to building domain-specific AI agents for data analysis tasks.
AWS launches Web Search on Amazon Bedrock AgentCore with tens of billions of documents, no external API required
Amazon Web Services launched Web Search on Amazon Bedrock AgentCore, a fully managed web search capability that gives AI agents access to tens of billions of documents without requiring external search APIs. The service, now generally available, runs entirely within AWS infrastructure and refreshes its index within minutes of new content appearing online.
Comments
Loading...