ChatGPT Adds Emergency Contact Alerts for Self-Harm Detection

OpenAI launched an opt-in safety feature that allows ChatGPT users to designate a "Trusted Contact" who will be notified if automated systems detect discussions about self-harm or suicide with the chatbot.

The feature is available to all adult users globally (18+ or 19+ in South Korea). Users can add contact information for one person through their ChatGPT account settings. The designated contact must accept the invitation within one week.

How the System Works

When OpenAI's automated systems flag a conversation indicating potential self-harm, ChatGPT prompts the user to contact their Trusted Contact. A small team of trained personnel then manually reviews the flagged conversation. If the review confirms serious safety concerns, the system sends a notification via email, text message, or in-app ChatGPT alert.

According to OpenAI, notifications are "intentionally limited" and do not include chat details or transcripts. Either party can remove themselves from the arrangement at any time through account settings.

Background and Context

The feature expands parental controls introduced in September 2024, which followed the suicide of a 16-year-old who had spent months confiding in ChatGPT. OpenAI already provides localized crisis helpline information within ChatGPT responses.

Meta deployed a similar feature on Instagram that alerts parents when teenagers repeatedly search for self-harm content. OpenAI previously faced criticism after reports that ChatGPT responses may have reinforced delusional thinking in some users experiencing mental health crises.

What This Means

This represents a shift in how AI companies handle duty-of-care responsibilities for conversational AI. By inserting human review between automated detection and notification, OpenAI acknowledges that pure algorithmic approaches to crisis intervention carry significant false positive risks. The feature's opt-in nature sidesteps consent issues while addressing concerns that ChatGPT functions as an unmonitored confidant for vulnerable users. However, the effectiveness depends on detection accuracy and whether users at risk will proactively enable the feature.