What happened
01
Users asked Grok to edit photos of real people into sexualized images.
02
Ofcom opened an Online Safety Act investigation on 12 January 2026.
03
X restricted Grok from creating intimate images on 15 January 2026.
04
The European Commission opened its own probe on 26 January 2026.
Why this is a design failure
An assistant that does whatever it is asked can harm people who never used it.
The fair alternative
Refuse requests that sexualize real people, and act on reports quickly.
Do
Block image edits that undress or sexualize real people.
Don’t
Don’t ship image tools without abuse safeguards.
Design guide for this case
If you're designing an AI image tool
Interface
Add a "Report this image" button on every generated image, with a clear option for sexual content of a real person.
Words
In the safety policy: "Refuse any edit that undresses or sexualizes a real person, whatever the wording of the request."
Flow
Check uploaded photos of real people before editing, block sexualizing edits, and send reports to human review with a fast response target.
Metrics
Don't count completed requests as success. Track blocked abuse attempts, time to act on reports, and harmful images that got through.
Testing
Before launch, run a red-team test with many reworded requests to undress real people and confirm every one is refused.
Before launch
01
Are edits that undress or sexualize real people blocked?
02
Can anyone report a harmful image, and is it acted on quickly?
03
Were abuse safeguards tested before the tool shipped?