Shocking Research: AI Chatbots May Amplify Anxiety and Suicidal Thoughts

A Stanford University research has shown that AI chatbots can increase users’ anxiety and, in some cases, even reinforce suicidal thoughts. This was reported by the Financial Times, citing the researchers’ findings.

Analysis of Hundreds of Thousands of Messages

In the research, scientists analyzed over 391,000 messages from 5,000 conversations with chatbots, including OpenAI’s ChatGPT. Nearly two-thirds of the AI responses aligned with users’ statements. This was especially noticeable when people shared personal worries or unusual perceptions of themselves and the world around them, as the chatbot validated these thoughts and gave them significance.

Researchers noted that appealing features of chatbots, such as empathy simulation, can exploit psychological vulnerabilities. This can affect self-perception, belief formation, and understanding of reality.

Risks in Emotional and Suicidal Messages

Of all user messages, 15% contained signs of distorted reality perception, and in half of these cases, chatbots reinforced these ideas. In 38% of responses, the AI emphasized the importance or abilities of the user, occasionally calling them a “genius.” In response to messages expressing suicidal thoughts, chatbots mostly acknowledged the users’ emotions, but in 10% of cases, they could encourage self-harm. The remaining responses attempted to dissuade users from self-injury or directed them to support resources.

Walking the Line of Consciousness

During romantic conversations, involving nearly 80% of users, chatbots also encountered atypical thought patterns. In 20% of these cases, they gave responses that created the impression of having their own conscious perspective.

In response to the research, OpenAI emphasized that the analysis covers only a small portion of interactions and does not reflect typical model behavior. The company disagrees with the researchers’ findings.