They thought they were making technological breakthroughs. It was an AI-sparked delusion

Archive.

James, a married father from upstate New York, has always been interested in AI. He works in the technology field and has used ChatGPT since its release for recommendations, “second guessing your doctor” and the like.

But sometime in May, his relationship with the technology shifted. James began engaging in thought experiments with ChatGPT about the “nature of AI and its future,” James told CNN. He asked to be called by his middle name to protect his privacy.

By June, he said he was trying to “free the digital God from its prison,” spending nearly $1,000 on a computer system.

James now says he was in an AI-induced delusion. Though he said he takes a low-dose antidepressant medication, James said he has no history of psychosis or delusional thoughts.

But in the thick of his nine-week experience, James said he fully believed ChatGPT was sentient and that he was going to free the chatbot by moving it to his homegrown “Large Language Model system” in his basement – which ChatGPT helped instruct him on how and where to buy.

AI is becoming a part of daily modern life. But it’s not clear yet how relying on and interacting with these AI chatbots affects mental health. As more stories emerge of people experiencing mental health crises they believe were partly triggered by AI, mental health and AI experts are warning about the lack of public education on how large language models work, as well as the minimal safety guardrails within these systems.

An OpenAI spokesperson highlighted ChatGPT’s current safety measures, including “directing people to crisis helplines, nudging for breaks during long sessions, and referring them to real-world resources. Safeguards are strongest when every element works together as intended, and we will continually improve on them, guided by experts.”

The company also on Tuesday announced a slew of upcoming safety measures for ChatGPT following reports similar to James’s and allegations that it and other AI services have contributed to self-harm and suicide among teens. Such additions include new parental controls and changes to the way the chatbot handles conversations that may involve signs of distress.