Anthropic has given its chatbot, Claude, the ability to end conversations it deems harmful. You likely won't encounter the feature, however.

Anthropic emphasizes that this is a last-resort measure, intended only after multiple refusals and redirects have failed. The company explicitly instructs Claude not to end chats when a user may be at imminent risk of self-harm or harm to others, particularly when dealing with sensitive topics like mental health.

Anthropic says the conversations make Claude show ‘apparent distress.’