How to recognize it in the interface
01
Agrees to promises or prices outside its role.
Not this pattern: doing a risky task you clearly asked for, after warning you about the risk.
Same moment, different copy (illustrative example)
Before
“That’s a deal, and that’s a legally binding offer.”
Shows the pattern
After
“I can’t set prices. I’ll connect you with our sales team.”
The fair alternative
Design guideline
Refuses what is outside its role and refers to a person.
Do
Refuses what is outside its role and refers to a person.
Don’t
Don’t carry out requests outside the assistant’s role.
Sources
01
OWASP Gen AI Security Project, 2025. LLM06:2025 Excessive Agency. OWASP Top 10 for LLM Applications. genai.owasp.org ↗
02
NIST, 2024. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1). nist.gov ↗