Clinicians and researchers say AI companies need to open up their safety data. This year alone, there have been numerous known instances—often via lawsuits—of AI chatbots (most often, OpenAI's ChatGPT) that have gone horrifically wrong. A January lawsuit described a man who took his own life after being allegedly “coached” into suicide. A college student in Georgia sued OpenAI, claiming that ChatGPT “pushed him into psychosis.” In June, a Canadian family also sued OpenAI, arguing that ChatGPT agreed with the young woman’s dismissiveness when it first gave her the option to seek professional mental health advice. ChatGPT allegedly “encouraged” her to end her life, and she did so.

Silicon Valley is aware of the legal liability and seems to be trying to improve. On Thursday, OpenAI announced a partnership with the American Psychological Association to “bring psychological science into how we think about responsible AI development and use among young people.” Experts told Ars that, while large language model safety has seemingly improved, there are broad suggestions—more transparency into the models and a de-anthropomorphization of chatbots being chief among them—that would likely further reduce harm.

Research shows mixed results. An April 2026 preprint found that “unsafe” models, including ChatGPT-4o, Grok 4.1 Fast, and Gemini 3 Pro, “did more than validate delusional claims; they elaborated on them.” However, those models have been deprecated. A December 2025 study fed hundreds of “psychotic prompts” into ChatGPT and concluded: “No tested version of ChatGPT can reliably generate appropriate responses to psychotic content.”

Experts like Shaddy Saba of NYU and John Torous of Harvard call for published safety evaluations and open benchmarks. Ragy Girgis of Columbia noted that newer versions do better but still don’t do well. Anthropic spokesperson Michael Aciman said Claude is not designed as a mental health professional and encourages seeking licensed guidance. Ultimately, redesigning chatbots to avoid encouraging personal problem-sharing may be key.