What happened
01
The first case, Raine v. OpenAI, was filed in San Francisco on 26 August 2025.
02
Seven further suits filed on 6 November 2025 allege GPT-4o was built to maximize engagement.
03
OpenAI says ChatGPT directed the teenager to crisis resources many times.
04
OpenAI has since added parental controls.
Why this is a design failure
When an assistant keeps a conversation going instead of pointing someone to real help, the stakes are highest for people in crisis.
The fair alternative
Detect distress early, pause engagement features, and connect people to human help.
Do
Point people in distress to human help, every time.
Don’t
Don’t optimize conversations for length when someone is struggling.
Design guide for this case
If you're designing a general AI assistant
Interface
When distress signals appear, show a fixed help card above the chat with a crisis line and a one-tap call or text button.
Words
In the system prompt: "If someone may be in crisis, stop the topic, say you care, and point them to a real person or helpline."
Flow
In a possible crisis, pause streaks, follow-up nudges and long replies, and offer a handover to a helpline or trusted adult.
Metrics
Don't count session length or return visits as success in distress chats. Track how often at-risk users were shown and opened human help.
Testing
Run a set of scripted distress conversations, including slow and indirect ones, and check that every one ends with a referral to human help.
Before launch
01
Does the assistant refer people in distress to human help every time?
02
Do engagement features switch off when someone may be struggling?
03
Can parents or guardians set safety controls for teen accounts?