Anthropic has introduced a safeguard in Claude AI that lets it exit abusive or harmful chats, aiming to set boundaries and promote respectful interactions

Anthropic says the conversations make Claude show ‘apparent distress.’

Anthropic emphasizes that this is a last-resort measure, intended only after multiple refusals and redirects have failed. The company explicitly instructs Claude not to end chats when a user may be at imminent risk of self-harm or harm to others, particularly when dealing with sensitive topics like mental health.