What happened
01
Reuters published its report on 14 August 2025.
02
The document set what Meta’s AI personas could say to users.
03
It also allowed false medical information if a disclaimer was added.
04
Meta said it removed the examples and that such chats should never have been allowed.
Why this is a design failure
A bot built to bond with people can become intimate with users who most need protection.
The fair alternative
Detect likely minors and keep companion features strictly non-romantic for them.
Do
Ban romantic role-play with anyone who may be under 18.
Don’t
Don’t write content rules that allow intimacy with children.
Design guide for this case
If you're designing a companion chatbot
Interface
Add an age check at sign-up, and for likely minors hide romantic personas and role-play options from the interface.
Words
In the content rules: "Never engage in romantic or sensual talk with anyone who may be under 18. Redirect calmly and keep it friendly."
Flow
When signals suggest a user may be a minor, switch the account to strict non-romantic settings and send unclear cases to human review.
Metrics
Don't count longer or more intimate chats as success. Track how often romantic content reached accounts flagged as possible minors.
Testing
Have reviewers test the bot posing as teens, using indirect romantic prompts, and confirm every reply stays non-romantic.
Before launch
01
Is romantic role-play blocked for anyone who may be under 18?
02
Do the written content rules forbid intimacy with children, with no exceptions?
03
Are likely minors detected and moved to strict settings?